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ABSTRACT 

The  Air  Force  (AF)  needs  an  evolving  software  tool  for  guiding  decision  makers 
through  the  complexities  of  software  outsourcing.  Previous  research  identified  specific 
outsourcing  strategies  and  linked  them  to  goals  and  consequences  through  a  variety  of 
relationship  rules.  These  strategies  and  relationship  rules  were  inserted  into  a  decision 
support  tool.  Since  that  time,  more  historical  data  and  outsourcing  literature  has  been 
collected  thus  necessitating  an  update  to  such  a  tool.  As  the  number  of  software 
outsourcing  projects  are  completed,  the  AF  must  capture  the  outsourcing  decision 
experiences  which  guided  the  projects  and  their  outcomes.  In  order  to  efficiently 
incorporate  this  new  experience,  the  decision  tool  must  be  redesigned  to  allow  the 
additional  knowledge  to  be  added  in  such  a  way  that  the  decision  rule  base  is 
automatically  updated.  With  this  new  feature,  the  tool  would  increase  its  precision  of 
predicting  software  outsourcing  success  as  the  software  outsourcing  knowledge  evolves. 
Capturing  software  outsourcing  as  knowledge  instead  of  raw  information  will  help  guide 
decision  makers  down  paths  proven  to  succeed  staying  clear  of  risks  that  historically 
plagued  software  outsourcing  projects  of  the  past.  Software  outsourcing  decision  makers 
desire  not  only  a  characterization  of  past  experiences  and  predictions  of  future  outcomes, 
but  also  reasons  to  help  them  make  informed  decisions. 
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SOFTWARE  DEVELOPMENT  OUTSOURCING 
DECISION  SUPPORT  TOOL 
WITH  NEURAL  NETWORK  LEARNING 

1.  Research  Introduction 

1.1  Problem 

The  Air  Force  (AF)  is  rapidly  losing  tacit  decision  knowledge  in  the  area  of 
software  development  outsourcing  as  experienced  decision  experts  leave  due  to  a 
declining  work  force.  Considering  that  knowledge  is  the  greatest  capital  of  an 
organization,  a  considerable  effort  must  be  made  to  capture  it  [1].  The  raw  data  of  past 
software  development  outsourced  projects  and  associated  outcomes  must  be  captured  and 
modeled  for  software  outsourcing  decision  makers  to  predict  solution  paths  for  future 
software  outsourced  development  projects.  Neural  networks  and  stepwise  regression  will 
provide  the  means  of  codifying  and  modeling  this  raw  data  into  knowledge.  Without  the 
use  of  such  learning  mechanisms  in  the  design  of  a  software  outsourcing  decision  support 
tool,  the  outsourcing  experience  data  will  go  unused  preventing  the  evolution  of 
outsourcing  costing  the  AF  money  and  other  resources. 

1.2  Background 

Under  current  policy  and  the  given  manpower  shortages,  software  outsourcing  is 
the  first  choice  for  USAF  software  development.  Thus,  most  Air  Force  software  project 
managers  chose  to  partially  or  wholly  outsource  their  software  efforts  with  little  thought 
to  realistic  outcomes.  Another  policy,  utilization  of  best  commercial  practices,  dictated 
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that  we  codify  historical  outsourcing  data  from  industry  and  government  projects  and 
apply  those  lessons  to  current  outsourcing  strategy  decisions. 

Air  Force  software  project  managers,  acquirers,  and  contract  software  developers 
would  benefit  from  a  continuous  evolution  of  practical  rules,  models,  and  a  tool  to  guide 
decision-making  as  it  relates  to  software  outsourcing  ventures.  As  future  project  data  is 
obtained,  the  tool  should  improve  its  ability  to  predict  outsourcing  consequences.  Also, 
capturing  software  outsourcing  knowledge  from  historical  outsourcing  data  will  help 
guide  decision  makers  toward  proven  outsourcing  strategies.  Future  decision  makers  will 
benefit  from  both  projected  outcomes  and  the  knowledge  behind  the  suggested  strategies. 

1.3  Thesis  Goal 

The  primary  goal  of  this  research  was  to  expand  existing  outsourcing  decision¬ 
making  support  rules  into  new  conventions  that  predict  software  outsourcing  outcomes. 
This  learning  process  must  be  incorporated  into  an  extensible  decision  support  tool 
designed  to  accept  a  flow  of  new  historical  outsourcing  data.  As  the  new  data  is 
processed,  this  tool  should  help  identify  and  discover  new  relationships  while  at  the  same 
time  quantify  the  usefulness  of  the  tool's  current  decision-making  rules  that  influence  and 
predict  software  outsourcing  consequences  and  results.  This  type  of  extensible  tool  will 
facilitate  knowledge  creation  and  future  outsourcing  strategy  decisions.  Neural  Networks 
(NN)  and  statistical-based  techniques  had  shown  to  be  extremely  useful  for  capturing 
such  tacit  knowledge;  thus  will  be  investigated  as  possible  learning  mechanisms. 
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1.4  Motivation 

Past  outsourced  software  projects  have  been  plagued  with  drastic  failures 
contributing  with  launching  vehicles  shooting  wildly  out  of  control,  battle  cruisers  being 
towed  back  to  port,  stealth  fighters  losing  access  to  their  target  support  software  due  to  a 
reboot,  Patriot  missile  system  falling  off-track  some  678  meters  causing  28  deaths  and  98 
injured,  etc...  [2,  3]  With  only  one  out  of  ten  projects  finishing  on  time  and  within 
budget  and  30%  of  government  projects  not  even  being  delivered  at  all,  software 
projects’  results  were  woefully  lacking  especially  considering  Department  of  Defense 
(DoD)  is  paying  $42B  annually  for  software  acquisitions  [2]. 

This  track  record  could  no  longer  be  blamed  on  technology  or  shortage  of 
computer  power.  Mosemann  and  other  leading  software  practitioners  viewed  software 
management  as  the  leading  cause  of  software  project  letdown  with  the  five  most  common 
software  management  failures  elaborated  in  Table  1  [4]. 

Table  1  Five  Main  Causes  for  Software  Project  Failure  [4] 

1 .  Unrealistic  project  schedule  with  no  true  way  to  estimate  development  time 

2.  Inappropriate  staffing,  not  enough  expertise,  or  unstable  staff 

3.  Changing  of  requirements  (many  essential  to  the  project) 

4.  Poor  quality  of  work  normally  brought  on  by  poor  quality  processes 

5.  Belief  that  lost  time  will  be  made  up  later  in  schedule  (reducing  testing  time) 

Senior  leaders  of  the  U.S.  Government  have  made  it  perfectly  clear  that  we  must 
improve  our  oversight  management.  President  Bush  publicly  endorsed  outsourcing  and 
insisted  that  our  top  executives  strive  to  apply  outsourcing  in  the  most  effective  manner 
utilizing  the  competitive  market,  providing  the  taxpayer  the  best  possible  fighting  force 
[5].  Secretary  of  Defense  Rumsfield,  at  the  2003  DoD  Budget  Briefing,  stressed  using 
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outsourcing  in  a  smart  and  efficient  manner  emphasizing  that  we  must  use  new  methods 
along  with  the  A-76  study  (outsourcing  efficiency  study)  to  correctly  apply  outsourcing 
[6].  In  1997,  Secretary  of  Defense  Cohen  stated  “we  still  do  many  things  in-house  that 
we  could  do  better  and  cheaper  through  outsourcing.’’  [7]  In  1994,  Secretary  of  Defense 
Perry  urged  DoD  to  adopt  current  civilian  best  practices  streamlining  software 
development  acquisitions  into  the  experienced  hands  of  commercial  outsourced  software 
developing  vendors  [8],  In  1996,  the  Defense  Science  Board,  backed  by  bills  in  both 
House  and  Senate,  mandated  cost  comparative  A-76  studies  to  compare  in-house  vs. 
outsourcing  for  providing  DoD  information  technology  supporting  functions  [7].  With 
this  senior  leadership  direction,  skills  and  knowledge  on  outsourcing  software 
development  must  be  evolved  to  correct  this  less  than  desirable  trend  [4].  In  a  Naval 
Analysis  Center  study,  Sam  Kleinman  researched  a  1000  A-76  studies  and  his  findings 
are  summarized  in  Table  2,  listed  below  [7]: 

Table  2  A-76  Findings  [7] 

1 .  Savings  were  not  from  outsourcing  but  from  reducing  number  of  workers 

2.  Three  percent  of  experienced  government  technical  workers  switched  over  to 
work  for  the  contractor 

3.  Contractor  audited  show  no  such  savings  over  government  workers 

4.  Cost  of  A-76  competition  was  not  included  against  projected  savings 

5.  Savings  represented  were  projected  savings  rather  than  actual  savings _ 

The  A-76  findings  were  unacceptable  and  improvements  crucial.  To  do  this,  one 
must  know  the  when's,  what's  and  how's  of  outsourcing.  The  experience  of  outsourced 
projects  must  be  captured  and  analyzed.  Given  the  amount  of  data  and  factors  relating  to 
this  experience,  analysis  techniques  must  be  explored  so  the  knowledge  can  be  fonned 
from  this  raw  data. 
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One  such  technique  involved  the  use  of  Artificial  Intelligence  (AI).  On  3  Feb 
1994,  the  USAF  Assistant  Secretary  sent  an  AF  publicly  distributed  letter  concerning  the 
use  of  AI.  This  letter  urged  the  research  and  studies  necessary  to  exchange  information 
and  improve  decision  making  [9].  With  this  motivation  to  seek  the  advantages  of  AI,  this 
thesis  will  focus  on  two  techniques  for  analyzing  this  software  outsourcing  data:  stepwise 
regression  and  NN  systems. 

1.5  Thesis  Document  Layout 

Chapter  Two  will  summarize  the  literature  researched:  1)  software  outsourcing 
and  2)  NN  and  stepwise  regression  techniques.  Upon  building  a  firm  understanding  of 
these  topics,  Chapter  Three  will  outline  a  methodology  plan  for  building  a  decision 
support  tool.  Along  with  this  plan,  measurements  and  validation  concerns  will  be  stated 
in  Chapter  Four.  Finally,  in  Chapter  Five,  conclusions,  issues  and  future  work  will  be 
given  so  that  this  complex  software  outsourcing  decision  making  knowledge  can  be 
evolved  parallel  with  the  evolution  of  software  engineering. 
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2.  Background 

2.1  Introduction 

This  chapter  includes  a  literature  review  helpful  in  establishing  the  foundation  for 
the  thesis.  The  research  is  separated  into  two  main  topics:  software  development 
outsourcing  and  neural  network/stepwise  regression.  Software  development  outsourcing 
is  broken  into  eight  sub-topics:  history,  outsourcing  differences,  benefits  and  drawbacks, 
strategies,  survey  results,  mitigation  efforts,  future  trends,  and  topic  conclusion.  Neural 
Network  (NN)/stepwise  regression  is  divided  into  six  sub-topics:  NN  history,  comparison 
studies,  NN  project  planning  concerns,  NN  feasibility  concerns,  NN  data  concerns,  and 
topic  conclusion.  The  chapter  concludes  with  a  summary  discussing  the  overall  findings 
and  results  of  the  literature  review. 

2.2  Software  Development  Outsourcing 

Outsourcing  has  been  defined  in  numerous  ways.  Washington  defined 
outsourcing  as  a  “contractual  agreement  between  a  customer  and  one  or  more  suppliers  to 
provide  services  or  processes  that  the  customer  is  currently  providing  internally,”  [7]  and 
Power  defined  it  as  “the  act  of  transferring  some  of  a  company’s  recurring  internal 
activities  and  decision  rights  to  outside  providers,  as  set  forth  in  a  contract.”  [10] 
Hennan  expanded  on  these  outsourcing  definitions  to  include  the  specifics  of  software 
development  outsourcing.  His  definition  was  broken  into  three  services  listed  in  Table  3 
[11]. 
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Table  3  Software  Outsourcing  Definition’s  Three  Services  [11] 

1 .  Development  of  complete  or  partial  software  products 

2.  Purchase  of  packaged  or  customized  package  of  software  products 

3.  Activities  to  aid  in  the  software  development  lifecycle  model 


Other  terms  associated  with  outsourcing  included: 

1.  Privatizing  -  far  left  subset  of  outsourcing  that  includes  the  transfer  of  facilities, 
equipment  and  other  government  resources  to  private  vendors  to  aid  in  the 
deliverance  of  the  outsourced  product, 

2.  Insourcing  -  augment  the  work  force  to  handle  work  load  (“rent-an-expert”), 

3.  Downsizing  -  eliminate  employees,  a  common  result  of  outsourcing,  and 

4.  Alliance  Outsourcing/Rightsourcing  -  use  correct  balance  of  in-house  and  outsourced 
resources  to  achieve  maximum  benefits  [7,  10-14]. 

2.2.1  History 

Outsourcing  has  had  a  distinguished  past  running  parallel  with  the  data  processing 
history  according  to  Ketler  and  Willems  [15].  In  the  1960’s,  large,  expensive  mainframes 
dominated  the  computer  world.  Software  experts  were  very  sparse.  Companies  were 
forced  to  outsource  data  processing  needs  due  to  these  limitations.  In  the  1970’s, 
computers  and  data  processing  became  more  powerful  and  less  expensive.  Few  of  the 
larger  companies  moved  data  processing  function  in-house  to  save  money.  Outsourcing 
still  remained  popular  due  to  the  lack  of  qualified  computer  experts.  In  the  1980’s, 
computer  cost  declined  while  their  popularity  and  usefulness  drastically  shot  upward. 
Majority  of  companies  concentrated  on  developing  their  own  in-house  Information 
Technology  (IT)  departments.  In  the  1990’s,  competitive  market  forces  and  in-house  IT 
expenses  forced  companies  to  IT  outsourcing  using  the  1989  Kodak  IT  outsourcing 
success  story  as  their  model.  In  the  2000’s,  after  various  outsourcing  studies  and 
research,  companies  faced  a  great  decision:  outsourcing  vs.  in-house  [15]. 
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2.2.2  IT  Outsourcing  vs.  Software  Development  Outsourcing 

Before  delving  into  the  factors  of  this  decision,  differences  between  IT  and 
software  development  outsourcing  must  first  be  established.  Power’s  thesis  [10]  included 
Kodak  and  Bell  South  Telecomm  (BST)  IT  outsourcing  case  studies.  Kodak  found 
certain  core  software  projects  were  best  left  in-house  while  successfully  outsourcing  non¬ 
core  software.  BST  echoed  similar  results  in  that  software  outsourcing  required  complex 
monitoring,  measuring,  and  planning.  Power  summarized  by  stating,  “Software 
development  is  not  easily  defined,  and  does  not  produce  easily  measured  outputs.”  He 
continued,  “Much  of  software  development  fills  the  planning  role  since  it  involves 
analyzing  processes  within  the  company  and  designing  software  programs  to 
accommodate  the  processes.”  His  thesis  showed  that  the  companies  studied  had  great 
success  with  other  IT  outsourcing  not  involving  software  development,  but  the 
complexities  involved  in  software  outsourcing  exposed  countless  problems  [10]. 

In  Hermann’s  software  outsourcing  dissertation  [11],  he  wrote:  “Software 
development,  however,  differs  from  most  outsourcing  because  companies  are  attempting 
to  contract  complex  intellectual  ‘project’  work  rather  than  typical  repetitious,  well 
understood  ‘process’  work.”  His  point  was  aimed  at  showing  how  IT  outsourcing  was 
different  than  project-type  software  outsourcing.  He  continued  by  stating,  “the  vast 
collection  of  IT  outsourcing  experience  literature  is  of  limited  value  to  a  customer  trying 
to  select  an  outsourcing  strategy  to  meet  an  organization’s  software  development  goals 
[11].” 
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In  another  DoD  outsourcing  case  study  performed  by  Washington  [7],  an  AFMC 
$87M  Software  System  saw  difficulty  in  meeting  and  identifying  essential  software 
requirements.  As  a  result,  the  outsourced  project  failed  acceptance  testing  costing  the  AF 
an  additional  $4.5M  to  finish  the  project.  In  Table  4  below,  Washington  stated  three 
factors  needed  for  a  strong  contractual  agreement  which  was  crucial  for  the  success  of  all 
outsourcing  ventures. 

Table  4  Strong  Contract  Factors  [7] 

1 .  The  products/processes  being  outsourced  must  be  completely  defined. 

2.  Considerable  effort  must  be  spent  to  measure  outsourcing  productivity  and  savings. 

3.  Fairly  accurate  time  estimation  methods  must  be  enforced. _ 

In  software  development,  these  three  factors  plagued  even  the  most  successful  projects 
which  strengthened  the  belief  that  software  development  outsourcing  must  be  considered 
differently  than  non-software  IT  outsourcing. 

2.2.3  Benefits  and  Drawbacks 

As  identified  above,  software  outsourcing  created  thorny  challenges  that  could  not 
be  easily  handled  as  other  IT  outsourcing.  Benefits  and  drawbacks  should  be  identified 
so  that  such  challenges  could  be  minimized.  Several  authors  noted  conflicting  beliefs 
between  benefits  and  drawbacks  [7,  8,  11,  13,  14,  16,  17].  Figure  1  summarized  these 
conflicts: 
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Outsourcing  Benefits 

Outsourcing  Drawbacks 

Reduce  Cost 

Increased  Cost 

Increased  Flexibility  (economy  of  scale) 

B-Team  syndrome  /  subcontracting 

Improved  Focus  ("do  what  we  do  best;  outsrc  the  rest") 
Increased  access  to  new  technology  (reduce  dev.  time) 

Complexity  of  outsourced  decision  / 
oversight  management 

Access  to  needed  expertise 

Contractual  overhead  /  litigation 

Sharing  risks 

Reversibility 

Improved  control 

Increased  risks 

Cash  flow  from  sale  of  intellectual  property 

Reduced  control 

Access  to  higher  CMM  level  processes 

Legal  ownership  of  intellectual  property 

Capture  knowledge  (knowledge  evolution) 

Loss  of  essential  org  skill  (knowledge) 
Security  /  confidentiality 

• 

Improved  product  quality 

Negative  work  force  psychological  impact 
Reduced  quality 

Figure  1  Outsourcing  Benefits  and  Drawbacks  (7,  8, 11, 13, 14, 16, 17] 

This  conflict  accurately  reflected  the  challenges  from  the  Power  and  Washington 

case  studies  [7,  10].  Software  outsourcing  required  complex  outsourcing  skills  not  found 

in  IT  outsourcing.  To  develop  these  skills,  software  outsourcing  must  be  dissected  into 

various  parts  and  assembled  into  a  model  that  can  be  comprehended.  The  goal  was  to 

maximize  the  required  benefits  while  minimizing  the  drawbacks  through  using  various 

strategies. 

2.2.4  Strategies 

Software  development  outsourcing  strategies  combined  several  aspects  into  a 
model  used  to  comprehend  the  decisions  that  must  be  made  to  ensure  such  benefits  are 
produced  while  keeping  the  drawbacks  to  a  minimum.  The  first  aspect  reminded  the 
decision  maker  of  the  different  facets  of  software  outsourcing.  The  relationship  of  these 
facets  are  summarized  in  Figure  2. 
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Figure  2  Software  Outsourcing  Facets  [13] 

People,  processes  and  technology  impacted  the  decision  according  to  several  references 
[11-13],  Software  projects  could  require  personnel  expertise  that  was  not  available  in- 
house.  Certain  Capability  Maturity  Model  (CMM)  level  processes  and/or  methodologies 
requirements  also  affected  this  decision.  Access  to  the  latest  technology  could  be 
required,  but  unavailable  in-house.  Each  facet  independently  or  collectively  influenced 
the  project  [1 1-13]. 

A  second  aspect  was  concerned  with  the  degree  or  type  of  outsourcing 
administered.  Figure  3  showed  the  variety  of  outsourcing  degrees/types: 


Figure  3  Software  Outsourcing  Scale  [11] 
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In  this  figure,  Hermann  illustrated  software  can  be  completely  developed  in-house, 
acquired,  or  somewhere  in  the  middle  with  product  and/or  process  outsourcing. 
Frequently,  Abbas  [13]  noted  that  projects  had  the  tendency  to  use  alliance  outsourcing  as 
shown  in  Figure  4. 


Software  Application 


PiPPP 

JJ 

A 


H  Outsourced 


□  In- 


House 


Figure  4  Software  Application  Outsourcing  Example  (13] 

Both  referenced  documents  showed  the  above  software  outsourcing  composition  [11,  13]. 

Each  component  could  be  completely  outsourced  or  developed  in-house.  Certain 
components  may  be  developed  partly  in-house  and  outsourced.  Another  variable 
depended  on  outsourcing  particular  processes  needed  to  develop  the  application  [11,  13]. 
The  decision  was  best  viewed  as  a  grid  of  possibilities  as  shown  in  Hermann’s  Figure  5: 
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Figure  5  Outsourcing  Percentage  Chart  [11] 

In  the  case  of  point  A,  roughly  1/3  of  its  in-house  processes  were  outsourced  while  1/4  of 
the  application’s  components  were  developed  in-house.  The  decision  possibilities  were 
endless  with  no  perfect  rule  to  follow  for  making  a  successful  decision.  Each  project’s 
management  climate,  in-house  expertise,  schedule,  goals,  and  other  similar  factors  will 
impact  such  a  decision  which  emphasized  the  need  for  a  decision  support  tool  [11]. 

A  third  aspect  separated  the  development  into  core  vs.  non-core  competencies. 
Both  Kodak  and  BST  found  certain  software  was  best  suited  for  in-house  development, 
while  non-core  software  could  be  successfully  outsourced  [10].  Regardless  of  the  type  of 
software,  the  software  oversight  management  should  remain  in-house  [13,  14].  Figure  6 
illustrated  this  relationship. 
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Figure  6  Competencies  Interaction  Diagram  1 13] 

The  decision  between  core  vs.  non-core  should  be  kept  in  the  software  managers’  fore¬ 
thoughts,  as  they  make  their  way  through  the  outsourcing  dilemma.  This  decision 
resulted  in  a  greater  amount  of  work  because  it  forced  management  to  perform  an  internal 
analysis,  a  feasibility  study,  and  a  market  forecast  as  shown  in  Figure  7. 
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Figure  7  Core  vs.  Non-Core  Decision  [  13] 

The  inputs  of  the  analysis  guided  the  software  development  decision  by  concentrating  on 
the  additional  trust,  safety,  and  control  factors  not  associated  with  non-core  software  [11- 
14].  This  added  credence  to  Mosemann’s  DoD  software  outsourcing  statement, 
“government  needs  enough  in-house  software  expertise  to  know  what  it  is  buying”  [4]. 

Taken  together,  these  aspects  and  the  desired  benefits  formed  an  outsourcing 
decision  model,  shown  in  Figure  8. 
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Outsourcing  Issues 

Ma  nagemcnt 

*  Capacity  availability.  capability  of 
outsourcing  vendors  ‘ 

*  Scale/size  of  outsourcing  (total/selective) 

*  Vendor  Relationship 
(strategic/  short  term! 

*  Contract  and  conformance 

*  Ownership.  IP,  &  location  of  soarce 

*  Frequency  of  use 

*  Bonding,  monitoring,  and  coordination 
efforts  and  resources 

*  Liability/  responsibility 

*  Flexibility  and  control 

*  Cost 
Technical 

*  Trust/  assa  ranee 

*  Technology  (Platform.  OS, 
development  tools,  programming  lang.) 

*  System  decomposition  and  interfaces 
“  Development  paradigms 

*  Process  visibility  (documentation, 
reports,  configuration  mangemeat) 

*  Maintenance  &  system  maintenahility 

*  Performance  measures 

*  Change.  CM.  and  evolvabOitv 


Figure  8  In-house  vs.  Outsource  Decision  Model  [13] 

The  figure  showed  the  required  internal  analysis  input  along  with  required  benefits  being 

fed  into  the  second  phase  of  studying  the  issues  associated  with  outsourcing  both  on  the 
management  and  technical  levels.  Management  issues  such  as  cost,  control, 
responsibility,  contract  and  outsourcing  degree  must  be  carefully  thought  out  along  with 
the  technical  issues  of  assurance/trust  (safety  levels),  technology  requirements,  system 
interface,  configuration  management,  maintenance,  and  similar  issues.  After  the 
collection  of  the  first  and  second  phase  information,  an  informed  decision  can  be  made 
(who,  when,  what  degree  and  go/no  go)  [13]. 

If  the  decision  was  made  to  outsource,  additional  outsourcing  knowledge,  as 
shown  in  Hermann’s  software  development  outsourcing  model,  Figure  9,  should  be 
understood,  so  that  outsourcing  outcomes  can  be  predicted  based  upon  relationship  rules 
built  upon  data  collected  on  previous  outsourced  projects. 


Taking  The  Decision 

|  Yes/No.  Who,  When,  What,  Degree | 

Key  Criteria 

*  Does  oulvourcing  have  good  fit  with 
company's  strategy  and  situation? 

*  Is  there  vendor/s  that  meet  company 
requirements? 

*  Will  it  help  build  a  copelilive  advantage? 

*  W ill  it  improve  performance/  time/ 

*  Is  it  feasible? 

*  Have  all  internal  alternatives  been 
evaluated? 


Company ' s  Si t ua tl on 

Key  Questions 


*  What  are  the  company's  strengths, 
weaknesses,  opportunities,  and  threats? 

*  I  loss  strong  is  the  company's 
competitise  position? 

I *  *  What  are  the  company  's  key 
|  competencies? 

Mow  far  does  change  is  the  industry 
I  affect  the  company's  business? 

*W  hat  are  the  industry's  dominant  traits?[ 
“  W  hicli  company's  are  in  the  strongest/ 
weakest  competitive  positions? 


Drivers  to  Outsourcing 

Kev  Factors 


*  Cost  saving 

*  Time 

(meeting  deadlines,  first  to  market...) 

*  Human  Resources  and  expertise 

*  Alliances  and  partenrships 

*  Technology 

*  Process 

*  Capability  development 

*  Specialisation  and  focus 

*  (flobalisalton 

*  Competition 
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Corporate  Software 
Development 
Strategy 

Markets 


Product  Level  Outsourcing  Strategy 

•  Processes  Outsourced 

•  Product  or  Component 
Types  Outsourced 

Customer  &  Supplier  Characteristics 

•  Location  (Proximity) 

•  Capabilities  (Track  record,  Domain, 
Tool,  Language  Experience,  Maturity) 


•  Government 

•  Commercial 


Product  Portfolio 

•  Domains 

•  Technologies 

•  Capabilities 


Relationship  Management  Policies 

•  Reviews  &  Visibility 

•  Inspections 

•  Payment  Arrangements 

•  Control 

•  Communication  Forms 

Project  and  Product  Characteristics 

•  Complexity 

•  Modularity 

•  Size 

•  Objectives 


c> 


/  \ 

Outsourcing 

Consequences 

\ _ / 


Figure  9  Software  Development  Outsourcing  Model  [11] 

The  similarities  between  both  models  in  Figure  8  and  Figure  9  were  clearly  evident. 


Each  model  took  the  various  factors  of  the  project  and  used  them  to  point  to  an  associated 
outcome.  Before  a  consequence  can  be  studied,  the  project’s  characteristics,  the 
relationship  management  policies,  the  contractor’s  characteristics,  and  the  degree  of 


outsourcing  information  must  be  known.  In  addition,  knowledge  about  the  company’s 


markets,  domains,  technologies  and  capabilities  will  help  categorize  such  information 


relating  to  the  outsourcing  consequences  [11]. 


2.2.5  Outsourcing  Survey 


In  the  section  above,  information  was  needed  to  predict  a  possible  outsourcing 


outcome.  Hermann  [11]  sought  to  do  that  very  thing.  He  constructed  a  survey,  included 


in  Appendix  G,  to  shift  our  focus  from  literature  knowledge  to  real  world  experience 
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knowledge.  Great  care  was  taken  to  create  a  survey  to  minimize  the  influencing  effects. 
Five  survey  goals  were  targeted  in  Table  5. 

Table  5  Survey  Goals  [11] 

1 .  Identify  outsourcing  decision  makers 

2.  Distinguish  which  type  of  software  development  is  most  common  (custom,  COTS, 
customizable  COTS,  or  none,  all  built  in-house) 

3.  Measure  which  style  of  outsourcing  is  used  most  often  (product,  process  or  both) 

4.  Recognize  which  outsourcing  goals  businesses  wished  to  achieve  and  found 
important 

5.  Summarize  software  outsourcing  relationship  rules  relating  to  consequences  and 

goals _ 


2.2. 5.1  Survey  Data 

Each  respondent  entered  a  vast  amount  of  information  categorized  in  the  following 


way: 

1 .  Personal  information, 

2.  Amount  of  software  outsourcing, 

3.  Experience,  domain  information  relating  to  their  last  outsourcing  venture, 

4.  Outsourcing  process  strategy  information,  outsourcing  product  strategy 
information, 

5.  Overall  project  goals  for  deciding  to  outsource, 

6.  Overall  project  consequences  or  outcomes  as  they  relate  to  outsourcing, 

7.  Information  relating  to  those  organization  and  contractor  roles  that  drove  the 
decision  to  outsource, 

8.  Information  relating  to  the  respondents  roles,  and 

9.  Project,  outsourcing  relationship,  outsourcing  expectation,  product  strategy,  and 
process  strategy  assertion  (methods  believed  that  increase  the  success  of 
outsourcing) 

Though  all  captured  types  of  information  are  important  to  help  understand  the 
outsourcing  experience,  Hermann  synthesized  the  data  elements  of  the  outsourcing 
domains,  strategies,  consequences,  and  assertions  into  his  Software  Outsourcing  Decision 
support  tool  version  1  (SODS1)  which  was  a  central  product  of  his  dissertation.  The 
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Boolean  variables  dealing  with  domains  and  strategies  were  known  to  the  SODS1 
application  as  input  variables.  The  consequences  and  assertions  were  categorized  as 
SODS1  output  variables.  These  variables  were  integer  type  data  being  built  using  a 
Likert  scale.  Each  input  and  output  variables  were  separated  into  individual  divisions  and 
were  included  in  Appendix  G.  In  Hermann’s  dissertation,  87  rows  of  data  (1  row  per 
each  survey)  were  collected.  The  survey  was  placed  in  an  interactive  website  allowing 
future  respondents  connected  to  the  internet  to  take  Hermann’s  survey  online  at 
http://www.eas.asu.edu/~outsrc/survey/.  There  were  48  new  survey  results  collected 
since  his  initial  research  concluded  [11]. 

2.2.5.1  Software  Outsourcing  Decision  Support  Tool  Version  1  (SODS1) 

S0DS1  was  designed  using  MS  Access  Visual  Basic  (VB)  with  an  interactive 
(point  and  click)  window  type  interface.  The  rules  that  were  created  and  validated  were 
then  inserted  into  S0DS1.  The  users  entered  their  input  data  described  in  the  survey  data 
section.  The  relationship  rules  used  this  data  to  calculate  an  output.  Such  output  was 
used  by  the  users  to  forecast  the  consequence  of  their  outsourcing  decisions  as  it  relates 
to  their  project  [11]. 

2.2. 5.3  Survey  Results 

Hermann  captured  outsourcing  demographics  goals,  goal  importance, 
consequences,  and  assertions  about  software  outsourcing.  This  work  established  a 
baseline  of  outsourcing  experience  beyond  anecdotal  case  studies.  A  few  highlights  of 
these  findings  were  discussed  below. 
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As  expected,  shown  in  Figure  10,  the  customer’s  project  manager  along  with  their 
corporate  culture  carried  the  most  decision  weight.  It  was  interesting  to  note  the  impact 
that  the  vendor’s  project  manager  and  company’s  culture  had  on  the  decision.  Questions 
emerged  from  conflicts  between  the  real  world  and  researched  materials.  Researched 
materials  implied  the  importance  of  technical  and  legal  support  [7,  8,  11,  13,  14,  16-18]. 
These  roles  were  listed  in  the  figure  but  took  a  back  seat  to  management’s  influence  [11]. 


Figure  10  Software  Outsourcing  Decision  Makers  [11] 

Next,  Hermann  established  that  custom  software  development  was  the 

predominant  type  of  outsourced  projects.  The  research  also  distinguished  that  hybrid 
outsourcing  (the  combination  of  process  and  product  outsourcing)  was  most  prevalent. 
Specifically,  Figure  1 1  shows  the  frequency  of  outsourcing  software  process  components. 
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Figure  11  Process  Outsourcing  Measurements  [11] 

Actual  development  processes  were  more  commonly  practiced  vs.  support  processes. 


The  top  four  outsourced  processes,  coding,  design,  testing  and  maintenance,  were 
consistent  with  the  literature,  but  surprisingly,  requirements  and  specification  which  took 
about  30%  of  those  who  used  process  outsourcing,  ranked  higher  than  most  of  the 
support  processes.  This  directly  contradicted  what  the  researched  literature  identified 
[11]. 

Third,  Hermann’s  research  also  studied  which  goals  that  companies  found 
important  and  wished  to  achieve  with  outsourcing  shown  in  Figure  12. 
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Figure  12  Outsourcing  Goal  Satisfaction  [11] 

Without  the  valuable  T-test,  these  results  would  be  at  first  misleading.  Those  four  goals 
indicated  within  the  brownish  tint  failed  to  meet  the  95%  confidence  level  and  should  not 
be  considered.  Those  goals  considered  significantly  unimportant  were  identified  within 
the  blackish  tint.  The  red  number  (1-6)  shown  at  the  bottom  of  Figure  12  identified  those 
goals  that  were  found  significantly  important.  Pointing  out  such  importance  proved 
beneficial  in  determining  the  amount  of  effort  and  resourced  invested  in  such  goals.  It 
was  interesting  that  the  goal,  reducing  schedule,  was  found  to  be  the  second  highest  goal 
of  importance  but  could  not  be  considered  due  to  its  low  confidence  level.  The  other  five 
significantly  important  goals  showed  positive  signs  with  adding  people,  expertise  and 
quality  [11]. 

Finally,  the  consequences  of  outsourcing  software  projects  were  cataloged  in 
Figure  13. 
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Figure  13  Software  Outsourcing  Consequences  [11] 

Software  outsourcing  was  found  to  give  the  customers  more  flexibility  in  their  schedule, 

improved  product  quality,  and  increase  responsiveness  to  both  organization’s  and 
customer’s  objectives.  However,  it  was  found  to  worsen  the  risks,  cost,  schedule 
duration,  control,  overhead  and  visibility  [11].  As  given  in  the  above  sections,  literature 
conflicts  not  only  each  other  but  with  these  results  as  well  [7,  8,  1 1,  13,  14,  16-18].  The 
question  marks  at  the  bottom  of  the  figure  signified  the  consequences  that  were  unknown 
because  the  responses  were  either:  1)  exceedingly  neutral  and/or  2)  lacked  a  clear 
positive  or  negative  tendency.  Hermann  used  these  assertions  related  to  these  goals  as  a 
means  of  guiding  outsourcing  mitigation  efforts  [11]. 


33 


AFIT/GCS/ENG/04-16 


2.2.6  Mitigation  Efforts 

Given  desired  consequences,  certain  mitigation  efforts  may  be  employed  to  assure 
a  positive  software  outsourcing  environment  that  minimizes  poor  consequences.  These 
efforts,  for  positively  influencing  software  outsourcing,  were  taken  directly  from 
researched  literature  and  summarized  in  Table  6  [4,  8,  10-14,  17-19,  22-31], 

Table  6  Mitigation  Efforts 

1.  Legal  support  [11-14,  16,  27] 

2.  Clear  and  concise  requirement  and  testing  specification  [11,  14,  30] 

3.  In-house  software  expertise  [4] 

4.  Clear  and  concise  contract  and  Request  For  Proposal  (RFP)  [11-14,  16] 

5.  Intellectual  ownership  [16] 

6.  Solid  software  management  principles  [12] 

7.  Requirement  and  testing  specs  agreement  [8,  13,  14] 

8.  Visibility  and  control  oversight  management  measures  [8,  10-12,  14,  30] 

9.  Positive  attitude  and  relationship  management  [18,  3 1] 

10.  Joint  software  reviews  [25] 

11.  Informal  communications  (knowledge  management)  [13,  19,  24,  28] 

12.  Knowledge  scouts  [28] 

13.  Software  dimensions  alignment  [23,  27] 

14.  Estimation  and  metric  practices  [8,  10-12,  14,  30] _ 


The  first  effort  listed  was  concerned  with  the  legal  outsourcing  relationship.  Legal 
support  was  needed  to  provide  guidance  due  to  the  possibilities  of  involving  expensive 
court  litigation  and  to  the  fact  that  software  projects  were  plagued  with  failures  according 
to  [11,  12,  14,  16,  27],  The  outsourcing  oversight  management  team  created  two  crucial 
documents,  the  contract  and  the  RFP.  These  two  products  contained  high  level 
management  goals  for  using  outsourcing  as  well  as  providing  legal  guidance.  By  letting 
the  contractor  know  the  reason  for  outsourcing  as  discussed  in  the  RFP,  energies  would 
then  be  focused  on  the  same  goals  [11,  12,  14,  16,  27].  Requirement  specification  or  the 
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process  to  attain  such  clear  and  testable  requirements  should  also  be  stated  in  these 
documents  [14].  Formal  methods  and  function  point  analysis  provided  techniques  to 
ensure  a  clear  and  testable  process  [11,  14,  30].  In-house  software  expertise  support 
should  be  required  for  the  oversight  management  team  in  their  venture  to  produce  the 
contract  and  RFP  as  preached  in  Mosemann  [4],  The  contract  must  also  provide  binding 
arbitration  and  cost  reimbursement  clauses  that  define  penalties  and  award  incentives 
preventing  litigation.  Oversight  management  should  plan  on  handling  disputes  between 
customers  and  contractors.  Chain  of  responsibility/authority  and  dispute  management 
must  be  clearly  stated  to  handle  disputes  at  the  lowest  levels  [14].  With  nearly  30%  of 
outsourced  development  ventures  ending  badly  or  even  in  court,  legal  support  staff 
provided  the  oversight  management  team  the  best  litigation  prevention  by  ensuring  the 
RFP  and  contract  were  written  in  terms  to  protect  the  customer  [11,  14].  Intellectual 
property  and  security  concerns  should  be  addressed  to  prevent  disastrous  problems  later 
in  the  development  [16]. 

Other  common  outsourcing  mitigation  efforts  involved  solid  software  project 
management  methods  taught  throughout  the  universities.  In  addition  to  these  taught 
principles,  testing  standards  must  be  planned,  developed  and  agreed  upon  by  both  in- 
house  software  experts  and  outsourced  vendors.  Depending  upon  software  criticality, 
quality  and  safety  concerns,  an  Independent  Verification  and  Validation  (IVandV) 
contractor  would  serve  as  a  watch  dog  protecting  the  interest  of  product  quality  and 
customer  concerns  [8,  13,  14]. 
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Metrics  offered  the  oversight  management  team  visibility  into  the  development  of 
the  software  and  some  control  over  the  final  product.  However,  metrics  were  plagued 
with  several  cautions.  They  were  expensive,  require  in-house  analysis,  were  easily 
misleading,  and  should  be  included  in  the  contract  and  RFP.  Even  with  these  cautions, 
metrics  provided  ways  to  track  progress,  schedule,  errors,  quality,  and  similar  project 
measurements  [12-14] 

The  oversight  team  must  establish  control  mechanisms  that  make  sense  to  the 
project  and  supported  by  the  in-house  software  experts  [4,  8,  13,  14].  With  only  one  out 
of  10  large  software  projects  completed  on  time,  within  budget  and  30%  of  projects 
failing  to  be  delivered  (those  delivered  containing  42%  of  the  planned  requirements), 
oversight  management  team  must  be  persistently  involved  [2,  14]. 

The  [18,  31]  articles  added  additional  focus  on  personnel  factors.  Positive 
relationship  and  attitude  management  ensured  communication  flows  freely  in  both 
directions.  It  eliminated  the  “We/They  Finger  Pointing”.  Outsourced  workers  often  had 
different  agendas;  therefore,  must  be  checked  especially  when  security  concerns  are  at 
risk.  Because  of  the  individualistic  goals  of  the  contractor,  care  should  be  taken  when 
assigning  them  to  work  with  in-house  teams  [18],  People  skills  should  also  be  carefully 
evaluated  along  with  the  technical  skills  when  selecting  a  contractor  to  interface  with  in- 
house  personnel  [31], 

Several  authors  showed  mitigation  efforts  aimed  specifically  at  software 
outsourcing.  These  efforts  were  included  in  Table  7. 
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Table  7  Software  Outsourcing  Specific  Mitigation  Efforts 


1.  Joint  software  reviews  [25] 

2.  Informal  communication  [13,  19,  24] 

3.  Knowledge  scouts  [28] 

4.  Software  outsourcing  dimensions  [23,  27] 

5.  Estimation  and  metric  clarification  [8,  10-12,  14, 

30] _ 


Joint  software  reviews  provided  an  important  communication  mechanism  for  the 
contractor  and  customer  to  come  together  and  to  identify  and  solve  issues  relating  to  the 
project.  Resources  must  be  spent  to  ensure  this  review  is  successful  and  the  review’s 
efforts,  energies  and  direction  were  aligned  for  all  parties.  To  accomplish  this  alignment, 
six  steps  were  listed  in  Table  8. 

Table  8  Joint  Review  Alignment  Steps  [25] 

1 .  Gather  all  meeting  members  goals  concerning  the  review 

2.  Create  shared  group  vision  and  goals 

3.  Collect  members  intentions,  preferences  and  justification  regarding  this  vision  or  goals 

4.  Discuss  consequences  of  not  meeting  goals 

5.  Prioritize  goals 

6.  Seek  group  consensus  to  press  forward  with  the  review _ 


It  was  found  that  informal  communication  decreased  as  the  distance  between  the 
customer  and  developing  team  increased.  When  this  distributed  (long  distance)  software 
outsourcing  was  compare  to  co-located  software  outsourcing,  the  co-located  development 
finished  1.5  to  two  times  faster  than  the  development  spread  out  over  distance.  The 
reason  for  this  dealt  with  the  decrease  in  infonnal  communication  and  coordination  [24]. 
In  some  instances,  such  as  the  need  for  certain  types  of  software  experts,  this  distributed 
software  outsourcing  was  required;  therefore,  the  following  informal  communication 
efforts  listed  in  Table  9  should  be  implemented: 


37 


AFIT/GCS/ENG/04-16 


Table  9  Informal  Communication  Efforts 

1 .  Instant  messaging  systems  such  as  on-line  chat  seemed  effective  at  building 
infonnal  neighborhoods  where  knowledge  transfer  was  found  to  increase  [24] 

2.  Knowledge  maps  gave  the  developers  an  expert  guide  to  who  knows  what  (much 
like  the  yellow  pages  for  the  outsource  development  program).  This  decreased  the 
time  needed  to  solve  coordination  problems  [19,  24,  32] 

3.  Identifying  and  building  communication  channels  between  experts  that  share 
common  interest  such  as  community  of  practices  provided  a  synergy  effect  of 
knowledge  creation  vital  to  the  development  effort  [24] 

4.  Web-based,  shared  calendars  considered  simple  but  effective  at  eliminating  some 
of  the  basic  coordination  issues  [24] 

5.  Knowledge  scouts  have  proved  useful  in  increasing  informal  communications  [28]. 


According  to  [28]  article,  knowledge  scouts  were  found  crucial  in  increasing 
informal  communications.  A  knowledge  scout  is  a  highly  energized  team  focused  on 
visiting  the  external  contractual  organizations  to  share/transfer  knowledge.  Face  to  face 
informal  meetings  provided  trust,  confidence,  energy  and  a  team  atmosphere  that  was  not 
available  through  video  teleconferencing.  Both  contractual  organizations  must  be  willing 
to  spend  additional  resources  for  this  method  to  work.  The  following  requirements  listed 
in  Table  10  will  provide  guidance  for  building  an  effective  knowledge  scout  program: 

Table  10  Knowledge  Scout  Program  Requirements  [28] 

1 .  The  team  must  know  key  project  development  information. 

2.  The  team  is  rewarded  for  being  alert,  productive  and  active. 

3.  They  must  plan  and  coordinate  several  visits  to  all  associated  players. 

4.  Knowledge  scouts  require  resources  and  a  special  brand  of  openness  therefore 
the  program  must  be  stated  in  the  contract  and  associated  RFP. 

5.  Each  member’s  paycheck  should  be  based  on  the  amount  of  knowledge 
actually  transferred. 

6.  The  team  should  have  an  outgoing  personality  and  people  skills. 

7.  Members  should  be  permanent  because  familiar  faces  seem  to  promote  trust 
and  confidence. 

8.  The  knowledge  scout  program  along  with  its  visits  should  start  early  in  the 

_ project  to  escape  the  investigator  label. _ 
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References  [23,  27]  stated  that  software  development  dimensions  must  be  aligned 
because  of  the  different  goals  and  cultures  represented  by  both  the  customer  and 
contracting  organization.  Six  software  development  dimensions,  also  known  as  the 
COCPIT  dimensions,  were  explained  in  the  [23]  article  and  summarized  in  Table  11. 

Table  11  COCPIT  Software  Development  Dimension  |23] 


1.  Coordination/control  focuses  on  different  cultures  between  the  companies. 

2.  Objectives  must  be  able  to  synch  as  well.  Many  contractors  wanted  to  make  a 
fast  profit  while  the  project  needed  a  certain  high  degree  of  quality.  Contractors 
in  an  effort  to  make  more  profit  (their  objective),  assigned  a  B-team  (a  team  with 
less  expertise)  to  certain  less  lucrative  projects  so  that  their  A-team  could  capture 
a  large,  expensive  project.  This  was  also  the  case  when  the  contractors  try  to  use 
subcontractors.  Contract  arrangements  should  account  for  some  control  in  such 
events. 

3.  The  project  demands  a  certain  type  and  amount  of  Capabilities.  Having  the  skills 
not  available  and/or  tied  up  in  other  projects  adversely  impacts  the  project. 
Selection  of  vendor  should  be  based  on  capabilities  not  lowest  bid. 

4.  Does  the  vendor  define  their  methodology  and  Processes?  The  Capability 
Maturity  Model  (CMM)  measure  the  maturity  of  such  processes.  The  project  due 
to  its  size  and  complexity  may  require  a  level  3,  4  or  5  CMM  vendor.  The 
oversight  management  team  maturity  level  must  also  align  well  with  the  selected 
vendor  or  project  frustrations  might  occur  due  to  the  gap  between  the 
organizations  maturity  levels.  Methodologies  also  need  to  be  judged  between 
both  teams. 

5.  Information  communication  was  discussed  in  the  earlier  paragraphs.  The 
oversight  management  teams  should  judge  and  select  a  vendor  based  on  similar 
means  of  communicating  and  a  willingness  to  communicate. 

6.  Both  the  software  and  development  effort  demands  a  certain  level  of  Technology. 
The  software  may  require  distributed-type  qualities  that  influence  how  the 
software  is  built.  Security/safety  requirements  influence  both  how  the  software 
may  be  built  and/or  the  development  nature  of  the  software  due  to  the  software 
security  classification.  Also,  technical  concerns  should  focus  on  the  digital 
communication  needs  between  both  customer  and  vendors. 


In  support  of  the  first  COCPIT  dimension,  Air  Force  Times  recently  printed  an 
article  about  Boeing  losing  $1B  due  to  integrity  issues  which  fell  out  of  alignment  with 
the  AF  expectation  [33].  In  an  extreme  programming  success  article  supporting  the  4th 
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COCPIT  dimension,  case  studies  showed  how  important  it  is  for  both  teams  to  be 
accustomed  to  the  methodology  (processes)  before  the  projects  starts  [29].  In  another 
article  supporting  such  dimension  it  was  found  that  certain  methodologies  work 
extremely  well  for  certain  projects  of  a  certain  size  and  complexity  and  not  so  well  for  the 
other  software  projects  [22], 

Rollo  and  Wright  urged  size/cost  estimation  and  metric  mitigation  efforts  focus 
their  aims  at  providing  a  greater  degree  of  control  and  visibility  into  the  project.  A 
planned,  structured  way  of  estimating  must  be  practiced  by  the  oversight  management 
team.  They  also  found  that  function  point  analysis  was  very  successful  at  judging  not 
only  the  size  and  complexity  of  a  project,  but  the  productivity  of  the  vendor  developing 
the  software  [30],  A  planned,  structured  way  of  using  metrics  must  also  be  practiced. 
References  [8,  10-12,  14]  noted  that  metrics  can  provide  vital  control  information  into  a 
project’s  schedule,  quality,  and  trouble  spots,  but  are  expensive  in  both  collection  and 
analysis;  thus,  they  must  show  significant  amounts  of  usefulness  and  be  stated  in  the 
contract. 

2.2.7  Future 

Along  with  mitigation  efforts,  the  future  trends  of  software  outsourcing  tends  to 
make  outsourcing  an  easier  choice  over  in-house  development.  The  literature  identified 
three  future  trends:  1)  Application  Service  Providers  (ASP)  [[34,  35],  2)  Partnerships 
[36],  and  3)  Knowledge  Management  [19,  32]. 

ASP’s  have  become  extremely  popular  in  the  software  outsourcing  world  [34,  35]. 
In  most  cases,  the  provider  supplied  not  only  the  software,  but  the  service  behind  the 
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software  such  as,  customer  support,  training,  integration  support,  database  access, 
internet  access,  backup  services,  recent  software  changes  or  version  management,  and 
server  hardware  support.  It  provided  the  customer  the  ability  to  lease  the  COTS  product 
versus  buying  or  building  the  software.  Four  drawbacks  for  buying/building  software 
were  summarized  in  Table  12. 

Table  12  Buying/Building  Software  Drawbacks  [35] 

1 .  Development  risk  in  building  or  customizing 

2.  Integration  problems  with  other  existing  customer  systems 

3.  New  technology  causes  endless  loops  of  software  evolution  meeting  customer’s  need 

4.  Development  or  customization  time  is  not  fast  enough _ 


ASP  had  several  benefits  over  buying/building  software  listed  in  Table  13  shown  below: 

Table  13  ASP  Benefits  [35] 

1.  30%  to  70%  Cost  savings  due  to  the  large  number  of  ASP  customers  sharing  the 
cost 

2.  Scalability  based  on  customers’  usage 

3.  ASP  specific  expertise  offers  faster  software  solutions  to  users 

4.  Promotes  distributed/mobile  workforce 

5.  Ensures  customers  have  access  to  the  best  practices  and  usage  of  latest  technology 

6.  Increases  help  support  from  other  ASP  subcontractors  due  to  the  importance  of  the 
large  ASP’s  account  to  the  subcontractors 

7.  Manages  one  ASP  outsourcing  relationship  versus  multiple  outsourcing 

relationships  with  multiple  vendors _ 


Susarla  [35]  summarized  the  findings  of  an  ASP  survey  including  over  250 
responses.  The  authors  found  that  the  biggest  benefit  dealt  with  speed  of  access,  quality 
support,  access  to  latest  technology  and  processes,  integration  with  other  systems,  and 
low  cost.  Many  (51%)  stated  that  they  were  up  and  fully  operational  in  under  a  month. 
ASP’s  do  have  several  drawbacks,  as  listed  below  in  Table  14: 
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Table  14  ASP  Drawbacks  |35] 

1 .  Not  suitable  for  core  (critical,  unique)  tasks  (80%) 

2.  Non-customizable 

3.  Compared  to  a  long  successful  custom  built  software  lifetime,  ASP  cost  can  be  higher 

4.  Dependent  on  oversight  management  to  monitor  performance  and  quality  of  service 

5.  Risk  of  interoperability  with  current  core  business  practices 


Several  authors  ASP  articles  included  a  list  of  cautionary  items  that  should  be  taken  prior 
to  signing  an  ASP  contract  [34,  35],  These  issues  were  summarized  in  Table  15. 

Table  15  ASP  Contract  Issues  [34, 35] 

1 .  Performance  and  quality  measures 

2.  Security  guarantees 

3.  Backup/disaster  guarantees 

4.  Legal  support 

5.  Implementation  plan 

6.  Required  training  and  user  support 

7.  A  plan  to  back  out  and  terminate  service  (data  transfer) 


According  to  Lee,  the  next  evolution  of  software  outsourcing  focused  on 
tightening  the  relationship  into  a  partnership  as  illustrated  in  Figure  14  below  [36]: 
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Figure  14  showed  that  most  current  software  outsourcing  relationships  were  built 
upon  the  formal  contract  (win-lose)  strategy.  An  outsourced  vendor  was  paid  even 
though  the  customer,  caught  on  the  losing  end,  found  the  delivered  product  worthless.  In 
a  partnership,  the  relationship  was  more  tightly  coupled.  Many  authors  stated  that 
outsourcing  relationship  was  like  a  marriage  making  a  partnership  relationship  like  a 
marriage  with  kids.  Both  companies  shared  resources,  personnel,  risks,  and  confidential 
concerns.  This  relationship  generated  higher  amounts  of  trust  due  to  a  win-win 
relationship,  a  relationship  important  for  developing  core  type  software  [36].  Examples 
familiar  to  the  AF  included  Federally  Funded  Research  and  Development  Centers  such  as 
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Aerospace  and  Project  Air  Force  (RAND)  corporations.  Both  parties  would  be 
negatively  impacted  if  the  relationship  fails  as  well  as  rewarded  if  the  relationship 
prospers. 

The  final  software  outsourcing  trend  was  identified  as  knowledge  management 
[19,  32],  These  articles  continued  with  changing  the  view  that  software  outsourcing  was 
an  acquisition  of  knowledge  instead  of  a  product.  Improving  business  processes  involved 
layers  of  various  skill-sets  connected  by  numerous  communication  channels.  Software 
development  took  this  same  approach.  The  development  normally  implied  an 
improvement  of  some  business  process.  Such  process  meant  knowledge  must  be 
captured,  codified  and  evolved  with  management  realigning  their  reasons  for  governing 
the  outsourced  vendor.  The  vendor  was  not  just  delivering  software  but  added 
knowledge  through  the  development  of  the  software.  This  realignment  affected  vendor 
selection  putting  more  emphasis  on  vendor’s  CMM  level  and  making  the  vendor’s 
methodology  more  important.  The  ability  to  attain  knowledge  from  the  documents  and 
training  was  viewed  as  important  as  ensuring  their  quality.  Other  software  outsourcing 
issues  pushing  knowledge  management  dealt  with  capturing  and  defining  the  processes 
used  by  software  acquisition  experts.  With  few  true  software  acquisition  experts,  the  ad 
hoc  processes  these  individuals  were  captured  and  defined  so  that  knowledge  was 
transferred  to  the  junior  acquirers.  In  Nov  2001,  DoD  experienced  its  first  federal 
acquisition  team  to  achieve  SA-CMM  level  two  [26].  This  team  was  the  US  Army 
Abrams  Project  Acquisition/Oversight  team.  Documenting  the  processes  of  how  an 
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organization  acquires  software  was  an  important  step  to  the  success  of  software 
outsourcing. 

2.2.8  Software  Outsourcing  Topic  Conclusion 

As  portrayed  in  the  above  graphs  and  figures,  software  outsourcing  appears  to  be 
very  complex.  Numerous  factors  relating  to  consequences  could  not  be  explained  in  an 
easy  formula.  Given  conflicts  in  the  literature  and  rapid  future  changes  in  software 
outsourcing,  literature  research  was  simply  not  enough.  Experienced  insight  proved 
important  in  capturing  decision  rationale  which  meant  capturing  the  factors  and  goals  for 
each  project  with  the  associated  outcomes.  From  this  experience,  a  standard  practice  of 
acquiring  software  development  would  emerge.  Through  analysis,  outcomes  could  be 
measured  in  terms  of  doing  worst,  neutral  or  better  given  mitigation  efforts  and 
associated  goals.  In-house  software  expertise  is  required  to  support  the 
acquisition/oversight  team  to  eliminate  reoccurring  mistakes.  This  captured  knowledge 
filtered  out  bad  practices  allowing  only  the  proven,  successful  outsourcing  methods  to  be 
kept. 

2.3  Learning  Mechanisms 

Given  the  need  to  capture,  codify  and  transfer  software  outsourcing  decision¬ 
making  knowledge,  research  offered  two  popular  methods  that  were  used  to  find 
significant  relationship  rules  or  patterns  so  that  given  enough  analysis  a  model  can  be 
constructed.  How  these  methods  provide  such  information  was  quite  different.  Stepwise 
regression  started  with  analysis  to  build  an  initial  model  along  with  the  interacting  data 
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affecting  the  outcome  [37].  Data  relating  to  this  model  was  collected  so  that  statistical 
analysis  could  be  made  to  validate  the  relationships  of  the  model.  The  second  step 
involved  adding  and  removing  predictor  variables  according  to  the  stepping  criteria.  The 
last  step  determined  tennination  based  on  either  the  exhaustion  of  search  possibilities  of 
the  stepping  criteria  or  when  desired  perfonnance  had  been  reached.  Stepwise  was 
broken  into  two  types  of  methods.  The  forward  stepwise  method  was  the  process  that 
started  with  the  least  number  of  effects  in  the  model.  Each  effect  was  statistically  tested 
against  an  entry  statistic  to  see  what  effects  could  be  added  to  the  model.  Immediately 
after  all  effects  were  added,  a  backward  removal  process  was  used  to  ensure  that  an  effect 
could  not  be  removed  based  upon  a  removal  statistic.  Backward  stepwise  method  took  an 
opposite  approach.  All  effects  were  designed  into  the  model  at  the  beginning.  Each  was 
statistically  tested  against  the  removal  statistic  to  decide  which  effects  could  be  removed. 
After  all  selected  variables  were  removed,  a  forward  entry  process  was  used  to  ensure 
that  a  variable  could  not  be  re-added  based  upon  the  entry  statistic  [37]. 

After  the  stepwise  regression  model  was  generated,  two  performance 
measurements  were  used  to  judge  how  the  model  related  to  a  given  set  of  observations. 
The  first  measurement,  the  correlation  (R)  value,  focused  on  how  well  the  model 
explained  the  variation  within  the  observations.  The  second  measurement,  the  Mean 
Squared  Error  (MSE),  was  used  to  determine  the  model’s  accuracy.  MSE  also  referred  to 
the  amount  of  noise  in  the  model.  Both  measurements  were  used  together  in  explaining  a 
model’s  performance  [38], 
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Several  authors  stated  NN  took  a  completely  different  approach.  It  utilized 
patterns  within  the  data  in  order  for  it  to  leam.  Many  definitions  were  offered  in  the  field 
of  computer  science  all  sharing  a  common  theme.  This  common  theme  considered  NN  as 
a  patch  work  of  many  connected  simple  processors  all  with  their  independent  amount  of 
memory.  These  connections,  considered  as  numeric  weights,  were  used  to  carry  input 
and  output  signals  useful  in  predicting  certain  outcome  patterns.  NN  could  seek  the 
advantage  of  each  processor  working  in  parallel  with  the  other  processors  thus  requiring  a 
great  deal  of  inner  connectivity.  However,  it  did  not  have  to  work  in  parallel  given  a 
simple  NN  model.  Also,  NN  involved  a  training  process  (learning)  where  the 
connections  and  processors  developed  some  learnt  weights  associated  with  input  and 
output  data  that  is  being  fed  into  it  [39-42],  Figure  15  presented  a  visual  diagram  of  two 
of  the  more  common  NN: 
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Figure  15  NN  Visual  Definition  [39-42] 

From  this  figure,  the  interconnecting  processor  acted  upon  the  inputs  and  outputs  that 
were  fed  into  the  NN.  As  the  NN  went  through  each  input  and  output  data  set  (patterns), 
the  processors  adjusted  its  weights  accordingly  not  only  to  the  current  pattern  but  from 
the  history  of  patterns  recorded  beforehand.  This  learning  was  broken  into  two 
categories:  1)  Supervised  learning  -  the  target  values  and  outputs  were  known.  Both 
were  fed  into  the  NN  during  initial  training  so  that  the  NN  could  establish  weights 
matching  its  outputs  with  the  inputs.  2)  Unsupervised  learning  -  the  correct  results  were 
not  known.  The  NN  was  fed  with  this  unknown  data.  The  NN  would  compress  and 
cluster  the  data  looking  for  connecting  patterns.  This  type  of  learning  was  usually  used 
for  prediction  type  functions  where  the  relationships  were  under  study  [39-42]. 
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Another  distinguishing  NN  category  depended  upon  allowing  learning  cycles  to 
develop  within  the  NN  [39,  41,  42].  If  cycles  were  allowed,  such  that  the  output  results 
were  fed  back  as  input,  then  the  NN  was  classified  as  a  feed-back  NN.  If  cycles  were  not 
allowed  the  NN  was  classified  as  a  feed-forward.  This  feed-back  required  a  long  amount 
of  time  before  obtaining  the  desired  learning  performance  measures.  This  drawback 
made  training  more  complicated,  but  may  be  required  given  the  type  of  data  and  the 
required  accuracy  of  the  NN  [39,  41,  42].  Both  methods,  took  advantage  of  historical 
data  better  known  as  backpropagation.  This  allowed  the  data  to  be  introduced  repeatedly 
into  the  NN.  Upon  each  presentation,  it  fine  tuned  its  associated  weights  corresponding 
to  each  pattern  [42]. 

Authors  stated  that  NN  could  be  separated  based  on  categorical  or  quantitative 
type  input  data.  Categorical  type  data  contained  a  finite  number  of  possible  values.  Both 
supervised  and  unsupervised  learning  could  be  associated  with  categorical  type  outputs 
called  classifications.  Quantitative  type  data  represented  as  numerical  measurements  that 
were  associated  with  some  arithmetic  relation  meaningful  to  the  data  and  associated 
output  [39,  41]. 

2.3.1  Neural  Network  History 

Along  with  this  definition,  the  NN  history  further  introduced  how  NN  had  evolved 
[39,  42,  43].  NN  existed  before  the  1950’s.  Relating  it  to  only  computer  science,  the  first 
neuro-computer  was  built  in  1954  by  Marvin  Minsky.  In  1956,  Dartmouth  established  a 
new  research  field  of  NN.  Shortly  after,  in  1957,  Frank  Rosenblatt,  with  the  support  of 
Cornell  University,  demonstrated  one  of  the  earliest  NN  systems  called  the  “Perceptron.” 
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This  system  was  capable  of  recognizing  letters  and  received  much  attention  until  1969 
when  the  Minsky  and  Papert  paper  discussed  the  limitations  associated  with  NN.  Given 
this  paper  and  the  limits  of  computing  power,  research  on  NN  came  to  a  stand  still  for 
over  a  decade  noted  as  the  NN  quiet  years  of  ’69  to  ‘82.  During  these  quiet  years,  expert 
systems  or  rule  based  systems  took  center  stage  in  the  artificial  intelligence  world. 
Numerous  expert  systems  were  built  trying  to  create  a  truly  intelligent  system.  These 
expert  systems  tended  to  require  large  amounts  of  programming  and  memory  to 
encapsulate  the  rules  and  decision  making  steps  implementing  those  rules.  Finally,  in 
1982,  Lecun  and  Papert  discovered  “backpropagation”  which  went  around  the  limitations 
discussed  earlier  in  the  1969  paper.  Immediate  following  this  discovery,  DARPA  started 
to  provide  NN  research  funding,  and  in  1986,  Rumelhart  and  McClelland  published  the 
“Parallel  Distributed  Processing”  book.  This  book  became  the  bible  for  which  several 
other  NN  topologies  were  designed  and  built.  Later  in  the  1980’s,  Hecht-Nielsen 
released  a  paper  that  disproved  the  limitation  associated  in  the  1969  paper,  and  the 
activity  of  NN  has  continued  to  grow  since  then  [39,  42,  43]. 

2.3.2  Comparison  Studies 

Both  NN  and  stepwise  regression  appeared  to  provide  methods  useful  in  capturing 
the  software  outsourcing  decision  making  knowledge,  but  the  question  was  does  one 
method  outperform  the  other  method.  Stepwise  regression  techniques  were  useful  in 
creating  a  metamodel  to  describe  simulation  modeling  as  given  by  the  [44]  article.  Both 
metamodel  and  simulation  modeling  were  popular  in  the  analysis  of  complex  systems. 
However,  simulation  modeling  involved  a  trial  and  error  process  in  order  to  flush  out 
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significant  relationships  used  in  predicting  a  set  of  outcomes.  If  the  relationship  did  not 
meet  the  significant  tests,  the  process  was  repeated  until  an  accepted  relationship  was 
found.  This  iterative  process  was  very  time  consuming  and  expensive  [44]. 

To  overcome  this  expense,  simulation  metamodels  were  used  to  establish 
relationship  rules  between  inputs  and  outputs  according  to  several  sources,  [37,  44-47]. 
These  relationships  could  span  over  wide  range  of  interests.  Upon  discovering  and 
validating  such  relationship  rules,  an  expert  system  could  be  created  using  such  rules  to 
solve  or  predict  an  outcome.  Stepwise  regression  was  one  of  the  more  common  methods 
of  flushing  out  such  relationships  based  upon  statistical  analysis.  Based  on  these 
relationships,  a  metamodel  could  be  built  to  present  how  the  relations  between  the 
significantly  important  variables  interact  to  flush  out  certain  outcomes  [37,  44-47]. 

Both  expert  systems  (built  upon  such  relationship  rules)  and  NN  were  used  to 
approximate  human  decision  making  process,  and  in  that  respect,  the  two  were  similar. 
How  they  performed  this  approximation  was  different.  Expert  systems  related  to 
conventional  data  processing  approach  in  that  detailed  programming  logic  must  be  used 
to  evaluate  the  rules  and  associate  the  rules  to  the  input  data.  This  meant  that  the  input 
must  be  complete  and  structured.  This  rigid  environment  often  lost  the  accuracy  because 
the  decision  making  process  involved  a  digital  like  yes  or  no  results  as  it  was  calculated 
from  predictable  program  logic.  Thus,  the  weakness  of  the  expert  system  was  the  rigid 
function  and  belief  that  the  knowledge  of  the  relationships  could  be  effectively  captured 
without  loss  of  accuracy  in  the  transformation.  However,  the  knowledge  of  such  rules 
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provided  some  insight  to  future  learning  and  explanation  of  the  problem  being  simulated 
[37,  39,  43-45,  48]. 

NN  on  the  other  hand  was  not  a  programmable  set  of  rules.  It  learned  from 
evaluating  the  input  and  output  data;  a  process  known  as  NN  training.  Given  this 
dependence  on  the  data  and  training,  great  care  would  be  taken  when  collecting  the  data 
and  using  the  data  to  train  the  NN.  The  NN  key  attributes  were  explained  in  Table  16. 

Table  16  NN  Key  Attributes 

1.  Learning  from  experience:  The  input  and  output  data  would  be  considered  the 
experience.  Since  the  NN  was  data  dependent,  it  used  experience  to  make  its 
choices  [39-41,  43,  46,  47,  49,  50]. 

2.  Ability  to  generalize:  Rigid  dependence  upon  rules  tended  to  be  very  specific  and 
unforgiving.  NN  could  take  input  data  that  was  unfamiliar  and  make  a  general 
detennination  based  off  the  experiences  learned  through  training.  As  well  as 
unfamiliar,  this  data  could  be  incomplete  or  noisy  (un-needed  data  that  is  not  useful 
in  making  the  decision)  [39,  40,  43,  46,  47,  49,  50]. 

3.  Compute  solutions  faster:  Having  to  based  decisions  upon  large  volumes  of  rules 
could  be  computationally  intensive  if  not  impossible.  NN  did  not  have  to  provide 
such  a  computational  task  because  of  its  ability  to  use  parallel  processing  between 
independent  working  neural  processing  models  [39,  41,  43,  46,  50]. 

4.  Less  reliance  on  the  expert:  Dependence  on  the  rules  meant  dependence  upon  the 
expert  or  source  of  the  rules.  Sometimes  this  expert  or  rule  discovery  given  such  a 
number  of  variants  was  not  available.  Also,  construction  of  those  rules  could  be 
very  demanding  and  tended  to  be  very  reliant  on  field  experience.  This  construction 
could  cause  the  introduction  of  various  errors.  NN  was  much  more  flexible 
providing  decisions  without  such  a  heavy  reliance  on  the  rules.  However,  some 
domain  expertise  was  required  for  choosing  the  correct  neural  network  design  and 
analyzing  the  input  and  output  data  [39-43,  46,  49,  50], 

5.  Non-linearity:  Development  of  rules  that  model  non-linearity  relationships  were  too 

complex.  Behavior  tended  to  be  non-linear  and  related  to  experience  which  was 
how  NN  attained  its  decision  making  capability  [39-43,  46,  47,  49,  50]. _ 


Tafti  [43]  contrasted  the  expert  “rule  based”  decision  making  techniques  with  NN 
and  found  that  both  are  needed.  The  shortcomings  of  one  technique  could  be  filled  by  the 
other.  In  several  articles,  both  techniques  were  used  to  support  each  other.  All  authors 
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strongly  encouraged  domain  and  data  analysis  prior  to  the  implementation  of  a  NN 
solution  [41,  43,  47,  49,  50].  Figure  16  summarized  this  NN  and  expert  system 
comparison. 


FEATURE 

EXPERT  SYSTEMS 

NEURAL  NETWORKS 

Ability  to 
handle  fuzzy  data 

No 
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Providing 

explanations 
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LOW 

Learning  by  example 

No 
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Self -adaptation 
to  new  situations 

No 

Yes 

Programming  effort 

High 

Low 

Capability  of  processing 
large  data 

High 

Low 

Figure  16  NN  vs.  Expert  Systems  |43] 


Several  references  introduced  how  both  NN  and  stepwise  regression  techniques 
could  be  used  together  to  build  metamodels  [44-46,  49].  The  most  popular  metamodel 
technique  was  regression  [43].  When  an  exhaustive  amount  of  relationships,  some 
possibly  even  hidden,  NN  was  used  to  discover  information  needed  in  creating  the 
metamodel.  Stepwise  regression  then  was  used  to  validate  this  information. 
Constructing  such  a  metamodel  in  this  backwards  approach  captured  the  strengths  of  both 
techniques,  but,  this  approach  may  introduce  numerous  errors  transitioning  from  one 
technique  to  the  other  [44,  46].  A  comparison  was  done  comparing  metamodels  built  by 
both  techniques  [44].  The  modular  function  NN  using  the  delta  learning  rules  performed 
the  best  when  compared  to  the  following  NN  listed  in  Table  17: 
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Table  17  Comparison  Study’s  NN  [44] 

■  3  General  regression  NN 

(each  built  with  Projection,  City  Block  and  Euclidean  summation) 

■  3  Radial  Basis  Function  (RBF)  NN 

(each  built  with  Projection,  City  Block  and  Euclidean  summation) 

■  4  Modular  function  NN 

(each  built  with  Delta  Rule,  Delta  Bar  Delta,  Quickprob  and  Maxprob  learning  rules) 

■  4  Backpropagation  NN 

(each  built  with  Delta  Rule,  Delta  Bar  Delta,  Quickprob  and  Maxprob  learning  rules) 

■  1  Learning  vector  quantization  NN 

(built  with  unsupervised  learning  Kohonen  mapping) _ 


In  this  study,  each  NN  outperformed  the  stepwise  regression  model  in  almost  every  case. 
It  was  found  that  these  summation  functions  do  affect  the  outcomes  in  the  general 
regression  NN.  Along  the  same  lines,  the  modular  function  and  backpropagation  NN 
were  found  to  be  sensitive  to  the  learning  rule  applied  and  the  amount  of  training  points 
used  in  training.  The  final  finding  showed  the  difference  in  training  a  NN  needs  in  order 
to  recognize  quantitative  vs.  qualitative  data.  The  report  found  that  two  separate  NN 
metamodels  would  be  needed  depending  upon  the  desired  output  data  [44].  Also,  in 
another  similar  test,  the  regression  model  was  compared  to  NN  model  [49].  NN  started 
to  learn  after  446  training  patterns  while  the  regression  technique  failed  to  even  establish 
a  relationship  after  15000  training  pattern  sets  [49].  Due  to  the  nature  of  NN,  it  has 
several  drawbacks.  In  Table  18,  these  drawbacks  were  summarized. 
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Table  18  NN  Drawbacks 


1.  First  drawback  includes  the  complexities  involved  in  training.  It  boils  down  to  trial 
and  error  with  a  major  reliance  on  experience,  intuition  and  domain  knowledge  [41, 
42,44,49]. 

2.  Second  drawback  relates  to  deciding  which  inputs  and  output  to  use.  Simplifying 
the  network  to  only  one  output  will  tend  to  be  faster,  easier  to  debug,  and  to  keep  the 
amount  of  errors  down  [41,  42,  44,  49]. 

3.  Third  drawback  deals  with  over- fitting  and  over-training  of  the  NN.  This  overfitting 
deals  with  the  complexity  of  associating  weights  to  a  wide  range  of  variants.  The 
number  of  hidden  weights  should  be  associated  to  the  number  of  inputs  and  classes 
of  those  inputs  used  to  train  the  network.  Over-training  is  training  the  NN  with  too 
similar  input  and  output  training  data.  The  noise  of  this  similar  data  will  be  learned 
causing  errors  during  the  NN  prediction  process  [[41,  42,  44,  49]. 

4.  Fourth  drawback  is  associated  with  its  black  box  nature.  The  user  accepts  the  output 

based  on  just  the  NN.  Validation  using  stepwise  regression  is  needed  to  associate 
the  outputs  into  meaningful  rules  [37,  39,  43]. _ 


2.3.3  Neural  Network  Project  Planning  Concerns 

Given  these  drawbacks,  certain  specialized  software  planning  must  be  made  when 
a  NN  is  involved  in  a  project.  Differences  that  must  be  kept  in  mind  when  planning  a  NN 
software  project  vs.  conventional  software  project  were  explained  in  Table  19. 

Table  19  NN  Project  Differences  [39] 

1.  NN  project  are  data  intrinsic.  Analyzing  and  collecting  the  data  needed  to  train  the 
NN  takes  considerable  amounts  of  experience,  time,  effort  and  resources. 

2.  Training  of  a  NN  boils  down  to  trial  and  error.  Building  prototypes  and  experimenting 
with  them  provides  the  only  assured  way  to  specify  the  correct  solution.  This  iterative 
trial  and  error  process  is  hard  to  control  thus  increasing  the  risk  for  budget  and 
schedule  over-runs. 

3.  Main  emphasis  of  a  NN  project  is  perfonnance  not  speed  of  processing.  Requirement 
specification  should  establish  a  minimum  performance  value.  Additional  emphasis 
should  be  spent  on  selecting  a  method  used  to  validate  and  test  these  performance 
measures. 


Several  authors  stated  project  planning  should  account  for  data  collection,  data  analysis, 
prototyping,  NN  training,  NN  design  and  perfonnance  reviews,  and  performance 
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validation.  Management  must  have  different  controls  to  allow  for  the  iterative  cycles 
required  for  trial  and  error  of  the  NN.  By  using  reviews,  management  can  put  some  type 
of  control  on  data,  prototype  cycle,  and  end  of  the  prototype  development  phase. 
Reviews  provided  a  way  to  align  energies,  effort  and  direction  thus  giving  management 
visibility  into  the  project  which  was  critical  to  ensure  the  project  is  on  time  and  within 
budget  [39,  41]. 

The  Figure  17  shows  a  typical  NN  software  project  life  cycle: 
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Figure  17  NN  Software  Lifecycle  Model  [39] 

The  project  lifecycle  was  separated  into  three  main  phases:  1)  application 
identification  and  feasibility  study  phase  2)  development  and  validation  of  prototype 
phase  3)  conversion  of  prototype  into  deliverable  software  system  phase.  Along  with 
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these  main  phases,  the  supporting  process  of  configuration  management  and 
documentation  had  their  own  respective  NN  issues.  Configuration  management  issues 
were  aimed  at  the  ability  of  keeping  the  data,  used  during  the  training  and  validation 
phases,  separated  and  backed  up.  Meta-information  regarding  the  prototype  must  be  kept 
separated  upon  each  experiment  and  prototype  cycle.  Documentation  issues  dealt  with 
knowledge  transfer.  The  ability  to  repeat  the  NN  trial  and  error  experiment  exactly  under 
the  same  conditions  was  crucial.  Results  and  details  were  recorded  along  with  any 
rationale  so  that  decisions  could  be  made  during  development  and  maintenance  phases 
[39,41], 

2.3.4  Neural  Network  Feasibility  Concerns 

In  the  first  phase  of  the  lifecycle,  the  application  should  lend  itself  to  a  NN 
solution.  Given  true,  the  second  phase  focused  on  the  expense  of  such  a  solution  by 
asking  feasibility  questions  such  as:  Has  another  similar  NN  project  been  accomplished? 
Is  the  data  required  available?  Do  all  the  resources  exist  to  collect  and  analyze  this  given 
data?  Upon  successfully  answering  such  questions,  business  feasibility  questions 
regarding  cost,  benefits,  and  risks  were  also  addressed  [39,  41]. 

Experience  and  published  examples  were  extremely  useful  in  deciding  what 
applications  lend  toward  a  NN  solution.  Table  20  listed  a  small  sample  of  applications 
where  NN  had  proven  quite  effective: 
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Table  20  NN  Application  Examples 

1.  Prediction  and  classification  problems  using  myeloma  survival  data  [51] 

2.  Deciding  factors  for  circuit  soldering  [47] 

3.  Predicting  optimal  termination  of  abrasive  air  flow  machining  in  air  intake  manifolds  [50] 

4.  Cost,  bond  rating,  stock  and  current  exchange  estimation  [39,  45] 

5.  Fault  diagnosis,  condition  monitoring,  forecasting,  signal/image  analysis,  pattern 

detection,  fraud  inspection  etc...  [39,  40,  45,  48] _ 


With  each  application  example  and  future  NN  applications,  three  criteria  were  applied  in 
order  for  NN  to  be  considered  as  an  appropriate  solution  for  the  project  [39,  41].  These 
criteria  were  listed  in  Table  2 1 : 

Table  21  Required  NN  Criteria  [39,  41] 

1 .  The  problem  can  not  be  defined  within  a  simple  set  of  equations  or  rules. 

2.  A  relationship  must  exist  between  input  and  output  data 

3.  Large  amount  of  suitable  data  exist  to  train  and  test  the  NN _ 


Along  with  these  requirements,  a  feasibility  study  was  performed  to  evaluate  the 
cost  of  a  NN  solution  with  its  benefits  [39].  In  project  planning,  it  was  necessary  to 
understand  the  cost  of  the  NN  was  more  than  other  typical  software  projects.  The  time- 
cost  to  iteratively  collect,  analyze,  train  and  test  the  prototypes  made  schedule  control 
extremely  difficult.  The  budget-cost  found  the  same  forecasting  pressures  as  well.  The 
additional  cost  of  data  collection  and  resources  required  in  analyzing  and  training  the  NN 
had  to  be  factored  within  such  a  budget  [39]. 

As  with  any  software  project,  sound  software  engineering  principles  demanded 
analysis  of  the  benefits  and  risk  mitigation.  Benefits  should  outweigh  the  concerns  of 
going  forward  with  a  NN  project.  Table  22  provided  some  common  NN  benefits. 


58 


AFIT/GCS/ENG/04-16 


Table  22  NN  Benefits  139] 

1 .  Reduced  staff  cost 

2.  Improved  decision  making 

3.  Enhanced  forecasting  support 

4.  Increased  monitoring  leading  to  better  perfonnance 

5.  Improved  fault  detection 

6.  Increased  knowledge  with  proper  analysis  of  results  increased  knowledge 


On  the  sunrise  of  these  benefits,  beware  of  the  storm  clouds  of  the  associated  NN  project 
risks.  These  risks  included  but  were  not  limited  to  those  risks  listed  in  Table  23: 

Table  23  NN  Risks  [39] 

1 .  Inability  to  get  required  performance 

2.  Unavailable  expertise  to  lead  NN  project 

3.  Benefits  blinds  management  feasibility  study  decision 

4.  Resistance  of  those  that  refuse  to  support  NN 

5.  Inabilities  to  collect  and  analyze  the  data 

6.  Increased  development  time 

7.  Increased  budget  cost _ 


The  mitigation  efforts  to  minimize  these  risks  were  found  in  one  word,  knowledge. 
Knowledge  in  literature,  expertise  and  past  solutions  involving  NN  paid  huge  dividends 
in  such  mitigation  efforts.  Along  with  this  knowledge,  the  application  of  sound  software 
engineering  processes  increased  the  success  of  a  software  development  project  [39]. 

2.3.5  Neural  Nework  Data  Concerns 

From  the  lifecycle,  it  was  obvious  that  data  was  a  major  part  of  the  NN  project. 
The  whole  push  behind  the  NN  solution  was  its  capability  to  take  different  types  of  data 
from  different  sources,  and  through  a  process  known  as  data  fusion,  produced  the  target 
output.  This  reliance  on  data  was  its  main  drawback  because  the  NN  will  only  be  as  good 
as  the  quality  and  appropriate  quantity  of  data  used  to  train  it  [39]. 
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As  stated  in  the  above,  the  amount  of  training  data  could  cause  error-prone  side 
affects  known  as  over-fitting  and  over- training.  One  of  the  advantages  of  NN  was  its 
ability  to  make  generalizations  about  input  data  to  formulate  a  respective  output,  better 
known  as  extrapolating.  In  order  for  the  NN  to  do  this  reliably,  it  must  first  interpolate  its 
training  data  to  a  best  fit  curve  or  weighted  system  necessary  to  calculate  a  certain  output. 
Through  data  analysis  and  stepwise  regression  models,  an  approximation  could  be  made 
concerning  the  amount  of  weights  needed  to  acquire  a  certain  output.  The  number  of 
training  input  sets  should  be  of  the  same  order  of  these  weights.  Other  experts  believed 
that  the  number  of  training  sets  should  be  of  the  same  order  as  these  weights  divided  by 
the  accepted  level  of  error.  Given  a  90%  confidence  level,  the  number  of  input  sets 
would  be  the  same  order  of  these  weights  times  10.  Because  this  approximation  was  not 
an  exact  science,  iterative  trial  and  error  training  prototyping  was  required  to  tune  the  NN 
[39,  45], 

Another  problem  associated  with  data  regards  missing  data  which  was  common 
after  data  collection.  Three  methods  were  used  to  handle  such  a  problem.  The  first 
method  simply  used  the  computed  mean  or  median  as  a  substitute  for  the  missing  value. 
The  second  method  required  capturing  this  missing  data  from  its  neighboring  sets.  It 
used  its  neighbor  values  to  fill  in  the  missing  data.  The  last  method  actually  used  a  NN 
or  a  step  wise  regression  metamodel  to  predict  the  value.  This  could  require  extensive 
amount  of  work,  but  given  the  importance  of  the  missing  data,  it  might  be  required  to 
minimize  the  error  given  such  a  substitution  [39]. 
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Ambiguous  input  data  also  appeared  to  cause  errors  if  not  identified  and 
appropriately  handled  prior  to  training.  This  type  of  error  resulted  from  the  same  input 
pattern  matching  two  different  outputs.  Special  allowances  must  be  made  to  ensure  this 
does  not  happen  especially  in  training,  validation  and  testing  [39]. 

The  type  of  NN  also  impacted  training.  In  the  comparison  studies,  it  was  shown 
the  differences  associated  with  the  type  of  NN  did  affect  the  outcomes  and  accuracy  of 
the  NN  [44].  Unsupervised  NN,  such  as  those  using  Kohonen  mapping,  did  not  require 
the  extensive  separation  of  the  data  into  training,  validation  and  testing  sets.  All  of  the 
collected  data  sets  were  used  in  training.  Data  analysis  was  still  required  for  validation 
purposes.  This  analysis  called  for  statistical  techniques  such  as  nonnalization  methods 
and  mean  squared  error  methods. 

However,  supervised  NN  separated  the  data  into  various  sets.  Two  of  the  most 
common  supervised  NN  architectures  were  Multi-Layer  Perceptron  (MLP)  (a  class  of 
backpropagation  NN)  and  Radial  Basis  Function  (RBF)  NN.  Each  of  these  two 
architectures  had  several  associated  data  concerns  regarding  each  [39,  40,  42,  45]. 

RBF  was  unique  because  it  involved  unsupervised  learning  along  with  supervised 
learning.  In  the  first  phase,  the  unsupervised  learning  portion  associated  a  selected  basis 
function  with  an  associated  width.  In  the  second  phase,  the  supervised  learning  portion 
associated  the  outer  layer  weights  to  the  identified  patterns  distinguished  in  the  first 
phase.  These  outer  layer  weights  gave  the  NN  ability  to  label  its  results.  Setting  the 
weights  was  a  linear  task  using  such  methods  as  the  least  mean  square  algorithm.  There 
was  no  validation  associated  with  RBF.  RBF  did  have  an  advantage  in  that  its  hidden 
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layer  was  accessible;  thus,  test  patterns  could  be  executed  to  determine  if  the  NN  was 
interpolating  vs.  extrapolating.  Training  times  were  slightly  shorter  for  RBF  vs.  MLP. 
MLP  had  a  higher  performance  in  generalization  and  classification  abilities.  Also,  RBF 
was  restricted  to  a  small  number  of  inputs  which  was  another  drawback  [39,  40]. 

MLP  (a  backpropagation  NN)  showed  to  be  one  of  the  highest  performing  NN  in 
the  comparison  studies  [44].  This  performance  came  at  a  price  because  validation  and 
testing  were  both  required.  Four  main  tasks  were  associated  for  this  type  of  NN  and 
listed  in  Table  24: 

Table  24  MLP  NN  Tasks  [39,  40| 

1 .  Partitioning  the  data  into  training,  validation  and  test  sets 

2.  Training  the  MLP  until  stopping  criterion  is  met 

3.  Selecting  the  optimal  network  based  on  validity  checks 

4.  Testing  the  trained  network  using  the  test  set _ 

Tarassenko  [39]  and  Smith  [40]  insisted  that  multiple  training  runs  must  be 
executed  against  the  NN.  This  was  in  part  to  its  need  for  a  random  weight  initialization. 
This  randomization  allowed  the  non-linear  optimization  of  the  associated  weight  sets. 
This  initialization  value  was  a  small  random  number  ranged  from  -0.01  to  0.01  and  used 
to  initialize  each  weight  within  the  outer  layer.  As  the  network  was  iteratively  trained, 
the  weight  sets  adjusted  appropriately.  Upon  each  iteration,  a  validation  or  a  stopping 
criterion  was  employed.  As  the  training  proceeded,  the  training  error  decreased  while  the 
NN  accuracy  increased,  and  the  same  was  true  for  the  validation  error;  up  to  an  error 
validation  minimum  point.  After  this  point,  the  validation  error  started  increasing  even 
though  the  training  error  showed  that  it  was  decreasing  [39].  The  following  plot  in 
Figure  18  illustrated  this  relationship  between  the  training  error  and  validation  error: 
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Figure  18  MLP  NN  Training  Error  Plot  [39] 

In  Figure  18,  an  epoch  was  defined  as  the  collection  of  all  patterns  given  in  the  input  data 
set.  It  could  take  hundreds  of  epochs  for  learning  to  occur.  With  each  epoch,  learning 
increased,  much  as  a  student  preparing  for  a  test  might  re-iterate  through  their  study 
material.  Learning  stopped  when  the  error  validation  minimum  point  had  been  reached 
as  displayed  in  the  above  figure  [39,  40]. 

Tarassenko  introduced  concerns  about  the  amount  of  data.  The  amount  of  data 
used  to  train  the  MLP  NN  should  be  known  because  this  same  amount  will  need  to  be 
used  to  validate  the  NN  and  again  used  to  test  the  NN.  This  1:1:1  ratio  was  very 
demanding  on  the  amount  of  data  collected;  therefore,  other  methods  may  need  to  be 
exercised  when  data  is  at  a  premium.  One  such  method  was  known  as  cross-validation 
which  maximized  the  amount  of  data  used  in  the  training  process.  The  original  collected 
data  was  split  up  several  times.  After  these  several  splits,  the  training  process  was 
performed  on  each  split.  For  each  smaller  split,  a  smaller  test  set  was  required  to  test  thus 
allowing  more  data  to  be  used  for  training.  An  example  of  this  was  given  in  Figure  19. 
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Arrangement  number:  1 
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•■Training  set 


►  Validation  set 


-  Test  set 


•  The  total  data  set  is  split  up  into  S  subsets,  each  of  equal  size. 

•  For  each  subset,  that  subset  is  chosen  to  be  the  test  set  and  the  other  S—  1 

subsets  are  combined  to  form  the  training  and  validation  sets. _ 


Figure  19  Cross-Validation  [39] 

Other  common  problems  associated  with  training  of  NN  were  summarized  in  Table  25. 

Table  25  Common  NN  Problems  [39] 

1 .  Incorrect  NN  solution  due  to  no  relationship  between  the  input  and  output  data 

2.  Incorrect  features  of  the  input  data  used  to  predict  a  certain  output  which  could 
cause  misleading  results  (another  reason  why  stepwise  regression  can  be  used  to 
support  findings  and  in  creating  a  metamodel) 

3.  Incorrect  initialization  values  (setting  them  to  close  to  0  or  1) 

4.  Incorrect  normalization  of  the  input  variables 

5.  Insufficient  number  of  training  patterns  found  in  the  input  and  output  data.  This 
causes  generalization  problems  and  could  lead  to  misleading  results 

6.  Incorrect  usage  of  NN  causes  extrapolation  vs.  interpolation.  This  could  happen 
when  the  NN  was  trained  for  predicting  a  certain  window  of  outcomes,  but 
someone  tries  to  predict  something  outside  the  window. 

7.  Incorrect  classification  results  due  to  unbalance  data  sets.  Expertise  is  needed  to 
analyze  the  data.  Given  only  few  patterns  relating  to  one  set  of  classification 
predictions  and  several  patterns  relating  to  another  set  of  classification  predictions 
could  cause  this  type  of  problem. 

8.  Over-fitting  and  over-training  problems,  as  previously  discussed  in  the  background 

section,  cause  poor  learning  performance.  These  two  problems  are  the  most 
common  errors  associated  with  NN.  Proper  analysis  of  the  data  and  carefully 
following  the  training  steps  will  prevent  such  problems. _ 


2.3.6  Topic  Conclusion 

NN  supported  a  variety  of  decision  making  type  software.  In  some  areas,  NN  were 
more  accurate  than  a  rule  based  system  created  from  stepwise  regression.  Stepwise 
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regression  assumed  linear  type  relationship  among  a  minimum  number  of  co-linearity 
variables.  As  the  complexity  increased,  as  the  number  of  variables  and  relationships  rose, 
and  as  the  relationship  tended  to  become  more  non-linear,  NN  proved  to  be  the  best 
decision  tool.  However,  the  importance  of  analyzing  and  understanding  the  data  was  still 
required.  This  comparison  showed  that  NN  and  stepwise  techniques  support  each  other 
to  draw  out  and  capture  the  knowledge  needed  to  make  future  decisions. 

2.4  Summary 

Future  software  outsourcing  decision  making  knowledge  must  be  captured  and 
modeled  so  that  knowledge  evolution  can  proceed  aiding  the  AF  in  saving  resources.  In 
order  to  represent  the  complexity  involved,  the  literature  has  given  us  several  models  to 
use.  NN  has  been  shown  to  be  useful  in  supporting  the  rules  associated  with  stepwise 
regression  techniques.  NN  projects,  however,  also  cause  additional  management, 
feasibility  and  data  concerns.  All  must  be  successfully  mitigated  to  ensure  the  software 
decision  support  tool  takes  proper  advantage  of  both  stepwise  regression  rules  and  NN. 
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3.  Methodology 

3.1  Introduction 

This  chapter  discusses  the  methodology  or  plan  for  survey  data  analysis  and 
building  a  software  outsourcing  decision  support  tool.  In  Chapter  Two,  it  was  discovered 
that  software  outsourcing  was  very  complex  along  with  numerous  factors  relating  to  its 
final  consequences.  The  literature  review  showed  many  conflicts  involving  software 
outsourcing  consequences  such  as  reduced  cost  vs.  increased  cost,  improved  control  vs. 
reduced  control,  and  reduced  schedule  vs.  increased  schedule  just  to  name  a  few.  These 
conflicts  were  summarized  in  Chapter  Two’s  Figure  1.  This  was  mainly  due  to  the  fact 
of  a  rapidly  changing  software  outsourcing  world.  Experience  proved  to  be  invaluable  in 
guiding  the  outsourcing  decision.  Also,  the  literature  research  uncovered  the  fact  that 
both  regression  and  NN  should  be  used  together  to  support  each  other’s  weaknesses.  The 
primary  goal  of  this  project  involved  taking  both  regression  and  NN  methods  and 
applying  them  to  all  of  Hermann’s  survey  data  collected  prior  to  2004.  These  learning 
mechanisms  captured  the  knowledge  hidden  deep  within  raw  survey  data.  Such 
mechanisms  provided  the  tool  an  efficient  means  for  displaying  this  knowledge  to  those 
outsourcing  decision  makers  seeking  such  support. 

3.2  Previous  Research 

Much  of  this  research  and  project  planning  was  built  upon  previous  work  that  was 
accomplished  in  Hennann’s  dissertation  titled,  “A  Decision  Tool  to  Support  Strategy 
Selection  for  Software  Development  Outsourcing  [11].”  The  product  of  this  research  was 
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a  survey,  analysis  methodology,  and  a  decision  support  tool  called  Software  Outsourcing 
Decision  Support  tool  version  1  (SODS1).  This  work  was  reviewed  because  much  of  the 
validation  of  the  new  tool,  SODS2,  rested  on  the  validation  of  SODS  1 .  Without  such  a 
solid  foundation  on  which  to  build,  this  methodology  would  be  severely  weakened. 

3.3  Research  Activities 

Given  the  foundation  listed  above,  this  section  covers  the  methodology  used  in 
data  analysis,  creation  of  stepwise  regression  models,  development  of  NN’s,  and 
development  of  SODS2.  Tool  selection  will  be  the  first  area  discussed.  Once  the  reader 
understands  how  the  tools  are  selected  and  evaluated,  the  new  survey  data  transformation 
and  analysis  will  be  introduced.  From  the  analysis  section,  the  two  learning  mechanisms, 
linear  regression  and  NN,  are  incorporated  in  this  design  for  the  creation  of  SODS2 
outsourcing  consequence  regression  models  and  NN’s.  Finally,  the  actual  plans  or 
designs  for  SODS2  are  presented  listing  the  software  requirements  that  must  be  met  to 
give  the  outsourcing  decision  maker  a  truly,  high  quality  decision  support  tool. 

3.3.1  Tool  Selection  and  Evaluation 

Using  this  foundation,  several  tools  by  default  had  to  be  used  in  the  development 
of  SODS2.  MS  Access  was  used  to  contain  and  organize  the  data  and  allowed  Open 
DataBase  Connectivity  (ODBC)  compatible  software  tools  to  access  such  data  through 
SQL  scripts.  MS  Outlook  was  used  to  collect  the  new  online  surveys. 

Hermann  made  reference  to  a  statistical  tool  called  SPSS,  “Statistical  Package  for 
the  Social  Sciences.”  Using  this  information,  along  with  suggestions  made  by  the  AFIT 
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statistical  department,  JMP  version  5.0  statistical  tool  was  used  in  applying  the  same 
statistical  methods  that  Hermann  used  with  SPSS.  The  tool  proved  to  be  a  good  fit  as  it 
connected  directly  to  the  MS  Access  database.  This  tool  also  provided  the 
implementation  of  T-tests  and  stepwise  regression  necessary  to  compare  the  results  of 
research  results  with  Hennann’s  previous  results  and  for  the  creation  of  the  consequence 
regression  models.  In  this  comparison,  the  tool’s  results  were  incorporated  into  MS 
Excel  charts  in  a  similar  manner  accomplished  in  Hennann’s  dissertation.  These  similar 
charts  delivered  the  analytical  overview  required  during  implementation. 

Researching  the  NN  tool  was  accomplished  in  the  same  manner  as  the  statistical 
tool.  The  evaluation  criteria  for  this  NN  tool  were  based  on  availability  and  the  tool’s 
ability  to  provide  20,  quality  NN’s  that  can  be  easily  accessed  by  SODS2. 

The  first  NN  tool  evaluated  was  Stuttgart  NN  Simulator  (SNNS).  To  design,  train, 
test  and  interface  the  NN’s  with  SNNS,  a  large  host  of  separate  components  were 
required;  most  of  which  were  written  for  the  UNIX  platform.  The  next  discouraging  find 
dealt  with  the  complex  script  language  required  for  SNNS  to  interface  with  SODS2. 

The  next  NN  tool  evaluated  was  Neurosolutions  4.0.  It  provided  a  graphical 
display  from  a  main  window  that  drove  everything  from  designing,  training,  testing  and 
interfacing  the  NN’s.  It  was  an  all  in  one  tool  that  had  no  compatibility  issues  running  on 
the  MS  Window  platfonn  which  made  fielding  SODS2  a  much  cleaner  approach.  Its  two 
highest  selling  points  included: 

1 .  Interfacing  was  delivered  through  the  use  of  Dynamic  Link  Library  (DLL)  files 
which  again  made  fielding  this  application  much  easier,  and 
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2.  It  came  complete  with  an  interactive  learning  demonstration,  NN  help  manual, 
and  an  excellent  technical  support  staff.  This  support  staff  provided  this  research 
NN  expert  guidance.  The  support  staff  made  Neurosolutions  the  choice  NN 
development  tool  for  this  research. 

MS  Access  VB  was  used  as  the  SODS2  developing  tool  largely  due  to  the 
Neurosolutions  tool.  Java  for  Togethersoft  was  first  conceived  to  be  the  tool  of  choice. 
Previous  Java  experience  showed  great  interoperability  with  other  components.  Java  was 
also  proven  to  work  well  with  ODBC  application  such  as  MS  Access.  Because 
Neurosolutions  directly  interfaced  MS  Access  VB  through  the  use  of  DLL  files,  Java  was 
not  needed.  MS  Access  VB  became  the  selected  developmental  tool  because  the  data  and 
application  could  be  contained  in  an  all-in-one  database  reducing  the  complexities 
involved  in  fielding  the  SODS2. 

3.3.2  New  Survey  Data  Transformation 

As  discussed  in  the  tool  selection  section,  MS  Outlook  was  used  convert  all  survey 
E-mail  messages  into  one  text  file.  Using  Togethersoft,  a  Java  applet  was  developed 
using  common  text  file  input/output  and  ODBC  commands  to  automatically  transfer  the 
48  text  based  surveys  into  48  data  rows  accepted  by  the  MS  Access  database.  After  the 
successful  completion  of  this  task,  the  new  database  should  have  a  total  of  135  rows  of 
survey  data  with  the  first  87  rows  comprising  the  old  surveys  collected  before  2000  and 
used  in  SODS1  and  with  the  last  48  rows  comprising  the  new  surveys  collected  after 
2000. 
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3.3.3  New  Survey  Data  Analysis 

The  new  survey  data  was  evaluated  as  it  relates  to  the  old  survey  data  reported  in 
the  dissertation.  In  Chapter  Two,  2.2.5  Outsourcing  Survey,  Hermann  introduced  many 
MS  Excel  charts  and  tables  on  outsourcing  assertions,  goal  importance,  goal  realizations, 
and  consequences  all  of  which  related  to  SODS  1 .  In  this  research,  these  initial  responses 
were  compared  to  the  old  data  to  detennine  the  amount  of  similarities  and  differences. 
The  analysis  included  the  following  statistical  comparisons:  mean,  standard  deviation, 
variance,  frequency  diagram,  and  T-test  at  95%  confidence  level.  The  exit  criteria  for 
this  phase  was  the  successful  completion  of  this  research’s  MS  Excel  charts  to  be  used  in 
comparing  with  those  created  in  the  dissertation  work. 

3.3.4  Regression  Model  Creation 

Once  the  similarities  and  differences  were  attained,  both  the  new,  old  and 
combined  survey  data  were  used  in  making  SODS2  consequence  regression  models.  It 
was  first  conceived  that  only  the  combined  survey  data  models  were  going  to  be  created, 
but  analysis  results  showed  greater  than  expected  differences  between  the  old  and  new 
data  sets.  SODS1  consisted  of  20  consequence  regression  rules  and  was  supported  by  an 
additional  14  goal  realization  regression  rules.  The  modeling  techniques  exercised  by 
this  research  was  evaluated  against  Hermann’s  regression  techniques.  Both  techniques 
used  stepwise  selection  procedures  with  an  entrance  statistic  of  95%  confidence  level  and 
removal  statistic  of  90%  confidence  level  [11].  The  exit  criterion  for  this  evaluation  was 
the  validation  of  this  research’s  modeling  techniques. 
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After  such  validations  were  met,  14  regression  models  for  the  outsourcing  goal 
realizations  were  created  and  analyzed  for  similarities  and  differences.  Once  satisfied 
with  the  analysis  and  realization  model  results,  60  regression  consequence  models  (20 
models  created  using  the  new  data  set,  20  models  created  using  the  old  data  set,  and  20 
models  using  the  combined  data  set)  were  carefully  designed  in  the  same  method  used  to 
create  the  14  goal  realization  models  with  the  exception  of  an  extending  stepwise  entry  / 
removal  statistics.  It  was  first  conceived  that  the  same  stepwise  statistics  used  in 
Hermann’s  models  would  suffice,  but  during  the  analysis  of  the  goal  realization  models, 
stepwise  current  configuration  settings  were  not  allowing  the  removal  of  any  model  input 
variables.  Extending  both  the  entrance  statistic  and  removal  statistic  to  75%  showed 
great  results  in  the  number  of  input  variables  allowed  to  be  inserted  into  the  model  and 
the  number  of  input  variables  that  were  first  inserted  and  then  removed  from  the  model. 
This  special  attention  to  detail  in  building  the  outsourcing  consequence  models  was 
significant  because  these  60  models  were  actually  encoded  into  SODS2  having  a  direct 
affect  on  the  output  of  the  tool.  Due  to  this  importance,  analyzing  such  differences  and 
similarities  not  only  validated  a  seamless  transition  from  SODS1  to  SODS2,  but  laid  forth 
a  means  of  allowing  the  outsourcing  knowledge  contained  within  the  models  to  evolve. 

3.3.5  NN  Development 

Chapter  Two  discussed  differences  between  learning  mechanisms  based  on 
regression  models  and  NN’s.  It  was  discovered  both  are  needed  for  this  application’s 
complete  development.  Also  from  Chapter  Two,  Multi-Layer  Perceptron  (MLP)  with 
backpropagation  and  Radial  Based  Function  (RBF)  NN  were  found  to  be  the  most 
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popular.  The  MLP  with  the  delta  learning  rules  showed  to  have  great  promise.  Its  main 
drawback  was  the  amount  of  data  required  for  training,  cross  validating,  and  testing. 
RBF  showed  promise  because  it  did  not  need  to  set  additional  data  aside  for  cross 
validation  [44].  As  a  result  of  these  finding  in  literature,  MLP  was  selected  to  be  the 
primary  option,  and  given  any  problems  with  not  enough  data,  RBF  would  be  a  backup 
plan. 

Neurosolutions  tool  selection  was  a  major  portion  of  this  NN  development.  This 
research  started  out  with  very  limited  NN  development  experience.  Neurosolutions 
demonstration,  user  manual,  and  support  staff  provided  this  research  the  NN  design, 
training,  testing  and  interfacing  results  recorded  in  Chapter  Four.  The  exit  criterion  for 
this  phase  was  the  successful  creation  of  20,  consequence  NN’s  built  to  some  degree  of 
accuracy  and  capable  of  being  interfaced  by  SODS2  during  runtime. 

3.3.6  SODS2  Development 

Along  with  the  new,  complete  survey  database,  SODS2  was  created  in  MS  Access 
VB.  Neurosolutions  provided  this  development  tool  a  working  sample  showing  how  this 
application  would  interconnect  SODS2  with  the  20  NN’s  explained  above.  This  was  the 
cornerstone  for  using  both  Neurosolutions  and  MS  Access  together  in  building  this 
application.  SODS2  will  need  to  meet  several  requirements  listed  in  Table  26  below: 
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Table  26  SODS2  High  Level  Requirements 

1 .  S0DS2  will  be  user  friendly  with  windows  and  point  and  click  commands 

2.  SODS2  will  allow  the  user  to  input  all  38  input  flags 

3.  SODS2  will  output  any  of  the  20  outsourcing  consequences  in  an  understandable 
report 

4.  SODS2  will  allow  the  user  to  select  a  desired  learning  mechanism:  NN  or  regression 
model 

5.  SODS2  will  contain  the  learning  logic  behind  the  60  regression  models 

6.  SODS2  will  interface  with  20  NN  by  passing  38  inputs  and  receiving  the  selected 
consequence  output 

7.  SODS2  will  output  an  assertion  report  based  on  the  user’s  outsourcing  projects  input 

8.  SODS2  will  allow  all  reports  to  be  printed  and  saved  as  a  text  file 

9.  SODS2  will  provide  instructions  and  help  tags  whenever  deemed  necessary _ 


SODS2  was  validated  through  the  usage  of  several  scenarios  created  directly 
from  surveys  collected  after  this  implementation.  These  surveys  were  not  used  in  the 
analysis  or  regression  /  NN  creation;  therefore  are  completely  new  to  such  research. 
Each  new  survey  provided  several  scenarios  matching  the  NN  and  regression  estimated 
output  against  the  actual  survey  outcome.  The  exit  criteria  for  the  SODS2  development 
phase  was  the  collection  of  the  validation  results. 

3.4  Summary 

This  chapter  described  the  methodology  for  supporting,  designing,  building  and 
testing  SODS2.  Key  principles  discussed  in  Chapter  Two  were  used  to  create  this 
research  methodology  and  SODS2  design.  Much  of  the  validation  from  Hermann’s  work 
was  implicitly  inherited  into  SODS2.  Since  SODS2  was  built  upon  the  latest  survey  data 
using  both  regression  models  and  NN  learning  mechanisms,  more  accurate  information 
was  available  to  the  software  outsourcing  decision  makers. 
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4.  Implementation  Results 

4.1  Introduction 

In  Chapter  Three,  a  methodology  for  analyzing  the  data,  creating  the  regression 
models  and  NN,  and  development  of  SODS2  was  presented.  This  chapter  presents  the 
outcomes  of  the  aforementioned  effort.  During  these  results,  the  methodology  changes 
documented  in  Chapter  Three  will  be  clearly  brought  forward  showing  the  additional 
support  associated  with  each  change.  Chapter  Three  also  discussed  the  baseline  from 
which  this  work  was  being  built  upon.  This  baseline  remains  unchanged  and  is  the  initial 
starting  point  for  this  chapter. 

4.2  Survey  Data  Transition 

As  stated  above  and  in  Chapter  Three,  the  new  survey  data  was  collected  through 
E-mail  messages.  The  old  survey  was  organized  neatly  in  a  MS  Access  database.  The 
goal  for  this  implementation  phase  was  to  transition  the  data  from  the  E-mail  messages  to 
the  database.  MS  Outlook  was  first  used  to  convert  the  48  new  survey  data  E-mail 
messages  down  to  one  complete  text  file.  Outlook  allowed  multiple  selections  of  all 
these  messages  and  the  “saved  as  text”  function  to  convert  such  selected  E-mail  messages 
as  one  text  file.  The  next  step  was  to  convert  the  text  file  to  database  records.  A  small 
program  was  written  using  Java  Togethersoft  to  provide  the  logic  and  ODBC 
connectivity  needed  to  complete  this  task.  An  ODBC  data  source  for  the  database  was 
created  using  MS  Windows  operating  system.  The  Java  ODBC  recognized  this  data 
source  and  gave  the  program  connectivity  to  the  database  by  means  of  SQL. 
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Transitioning  the  text  data  into  acceptable  MS  Access  data  elements  was  more 
challenging.  Several  string  parsers  and  associated  logic  were  designed  and  tested  to 
move  the  data  over  from  string  data  types  to  integer  data  types.  Upon  successful  type 
conversion,  a  host  of  input  SQL  commands  were  developed  and  tested  to  finally  convert 
the  readable  survey  data  into  database  records.  The  phase  drew  successful  conclusion 
with  a  new  database  containing  the  48  new  survey  responses  along  with  the  87  previous 
survey  responses. 

4.3  Survey  Data  Analysis 

This  phase  concentrated  on  analyzing  the  new  survey  data,  now  stored  in  the 
database,  against  earlier  survey  data.  The  purpose  of  this  analysis  was  to  determine  if  the 
new  survey  data  was  significantly  different  from  the  earlier  responses. 

4.3.1  Outsourcing  Experience  Analysis 

Using  JMP  5.0,  the  outsourcing  experience  data  was  analyzed  first  to  summarize 
the  difference  between  the  amount  of  respondent’s  software  development  outsourcing 
between  the  new  and  old  data.  The  old  survey  outsourcing  experience  statistics  were 
shown  in  Figure  20: 


Number  of 
Projects  per 
Respondent 

Percent  of 
Outsourcing 
within 
Respondent 
Organization 

Mean 

5.5132 

26.6753 

Median 

3.0000 

10.0000 

Mode 

2.00 

.00 

Std.  Deviation 

7.0569 

31.6712 

Figure  20  Old  Survey  Data  Outsourcing  Experience  [11] 

The  new  outsourcing  experience  was  listed  below  in  Table  27: 
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Table  27  New  Survey  Data  Outsourcing  Experience 


Number  of  Software 
Outsourcing  Projects  per 
Respondent 

Percentage  of  Software 
Outsourcing  Practiced  by 
Respondent’s  Organization 

Mean 

7.27 

38.34 

Median 

5 

30 

Standard  Deviation 

6.48 

27.16 

Standard  Error 

0.72 

4.40 

The  new  data  showed  that  recent  respondents  were  using  outsourcing  significantly 
more  than  previous  organizations.  Such  results  identified,  according  to  the  new  survey 
data,  that  software  outsourcing’s  popularity  and  positive  results  were  growing.  The 
results  also  indicate  the  new  survey  results  were  built  on  more  outsourcing  experience 
than  those  previously  captured.  This  was  a  harbinger  of  good  data  quality. 

4.3.2  Outsourcing  Project  Assertion  Analysis 

In  Hennann’s  dissertation,  a  percentage  chart  was  used  to  summarize  outsourcing 
assertion  data.  This  chart  was  reproduced  in  the  same  manner  except  showing  the  new 
survey  data  analysis.  The  survey  asked  for  a  variety  of  assertion  questions  regarding 
outsourcing  projects.  The  purpose  of  these  questions  was  to  distinguish  methods  or 
assertions  believed  to  help  make  the  outsourced  project  a  success.  Responses  ranged 
from  1,  “strongly  disagree”,  to  5  “strongly  agree.”  The  chart  simplified  the  response  by 
converting  responses  1-2  to  mean  “agree”,  3  to  mean  “neutral”,  and  4-5  to  mean 
“disagree.”  For  a  more  in  depth  explanation  of  each  assertion,  please  refer  to  Appendix 
G.  The  following  subsections  will  analyze  each  type  of  assertions. 
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4.3. 2.1  Outsourcing  Project  Assertion  Analysis 

Project  assertions  were  methods  set  toward  improving  outsourced  project  as  a 
whole  regardless  of  outsourced  strategy.  The  following  Figure  21  summarized  the 
project  type  assertions  for  the  old  data: 


Figure  21  Old  Survey  Data  Project  Assertion  Analysis  [11] 

The  new  survey  data  project  assertion  analysis  was  listed  below  in  Figure  22: 
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0%  - - - 1 - - - 1 - - - . - - - 1 - - - 1 - - - 1 - - - . - - - 

Vendor  Tool  Domain  Domains  Vendor  Reuse  Domain  Vendor  Large  Projects  Larger  Efforts 
Experience  Vendor  Influence  Better  Customer  Available  Better  Better 

Better  Experienced  Experienced  Better 

Better  Better 

Project  Assertion 

Figure  22  New  Survey  Data  Project  Assertion  Analysis 
The  results  were  similar  with  the  exception  of  the  larger  project  and  effort  assertions. 
The  new  data  showed  the  respondents  were  neutral  with  no  clear  tendency  going  either 
way  on  those  two  assertions.  A  possible  explanation  for  this  result  could  lie  with  project 
outsourcing  management.  Maybe  new  outsourcing  management  techniques  were 
allowing  software  outsourcing  success  for  larger  projects  and  efforts.  However,  since 
this  analysis  showed  these  two  assertions  as  neutral,  they  were  left  out  of  SODS2. 

4.3.2.2  Outsourcing  Relationship  Assertion  Analysis 

Relationship  assertions  were  methods  set  toward  improving  buyer  /  vendor 
relationship  and  much  like  the  project  assertions  affected  the  outsourced  project  as  a 
whole  regardless  of  the  outsourced  strategy.  Figure  23  summarized  the  relationship 
assertions  for  the  old  data: 
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□  Negative 
■  Neutral 

□  Positive 


Figure  23  Old  Survey  Data  Relationship  Assertion  Analysis  [11] 


The  new  survey  data  relationship  assertion  analysis  was  listed  below  in  Figure  24: 
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The  results  showed  a  decrease  in  every  outsourcing  relationship  assertion.  However, 
only  proximity  and  distance  assertions  failed  to  be  consistent  with  the  results  captured  in 
the  old  data.  Both  of  these  assertions  involved  distances  between  the  buyer  and  vendor. 
Proximity  referred  to  the  buyer  and  vendor  being  located  within  small  distances  while  the 
distance  assertion  referred  to  the  increased  worked  day  due  to  the  distances  separating  the 
buyer  and  vendor.  Respondents  felt  that  proximity  was  a  factor  in  outsourcing  success 
which  was  completely  reversed  in  the  old  data.  The  distance  assertion  received  a  mixed 
reaction  in  the  new  data.  Thus,  this  assertion  was  virtually  neutral  when  compared  to 
significant  disagreement  from  the  initial  research.  A  possible  explanation  for  both  results 
could  lie  with  the  increase  in  communication  tools  making  it  possible  to  have  a  virtual 
face  to  face  meeting  over  the  internet  and  ability  to  communicate  software  development 
information  needed  by  a  software  outsourcing  project.  Since  there  was  no  clear 
alternative  for  the  proximity  assertion  and  the  mixed  reaction  regarding  the  distance 
assertion,  both  assertions  were  removed  from  SODS2. 

4.3.2.3  Outsourcing  Project  Goal  Assertion  Analysis 

Project  goal  assertions  were  methods  set  toward  improving  outsourcing  goals  and 
much  like  the  project  assertions  affected  the  outsourced  project  as  a  whole.  Figure  25 
summarized  the  project  goal  type  assertions  for  the  old  data: 
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aggressive  schedule  worse  aggressive  cost  worse  aggressive  schedule  better  aggressive  cost  better 


Figure  25  Old  Survey  Data  Project  Goal  Assertion  Analysis  [11] 

The  new  survey  data  project  goal  assertion  analysis  was  listed  below  in  Figure  26: 


Figure  26  New  Survey  Data  Project  Goal  Assertion  Analysis 
The  results  were  somewhat  different.  The  new  data  showed  more  varying  opinion 

regarding  project  goal  assertions.  The  new  data  respondents  saw  aggressive  cost  project 
goals  as  a  bad  influence  on  outsourcing  success  by  a  two  to  one  margin.  However,  they 
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had  no  clear  tendencies  regarding  aggressive  schedule  project  goals.  This  trend  should  be 
investigated  to  determine  if  software  outsourcing  was  starting  to  offer  more  schedule 
flexibility.  As  a  result,  aggressive  schedule  project  goal  assertion  was  left  out  of  SODS2. 

4.3. 2.4  Outsourcing  Process  Assertion  Analysis 

Process  assertions  were  methods  set  toward  improving  outsourcing  where  process 
outsourcing  strategy  was  being  practiced.  These  assertions  were  aimed  at  the  interactions 
between  in-house  and  outsourced  processes.  Figure  27  summarized  the  process  type 
assertions  for  the  old  data: 


The  new  survey  data  process  assertion  analysis  was  listed  below  in  Figure  28: 
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Tool  Information  Interfaces  Same  Lifecycles  Vendor  Maturity  Customer  Maturity  Maturity  Close 
Flow  Better  Responsibilities  Better  Better  Better  Better 

Defined  Better 

Process  Assertion 

Figure  28  New  Survey  Data  Process  Assertion  Analysis 
The  results  were  very  similar.  Having  a  comparable  maturity  levels  between  vendor  and 
customer  showed  a  decrease,  but  with  a  near  to  two  to  one  ratio,  this  assertion  will  remain 
unchanged.  Since  this  analysis  showed  similar  results,  all  process  type  assertions  were 
included  in  SODS2. 

4.3.2.5  Outsourcing  Product  Assertion  Analysis 

Product  assertions  were  methods  set  toward  improving  outsourcing  success  where 
product  outsourcing  strategy  was  being  practiced.  These  assertions  were  aimed  at  the 
outsourced  development  of  a  product  or  sub-product.  Figure  29  summarized  the  product 
type  assertions  for  the  old  data: 
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Figure  29  Old  Survey  Data  Product  Assertion  Analysis  [11] 

The  new  survey  data  product  assertion  analysis  was  listed  below  in  Figure  30: 


Figure  30  New  Survey  Data  Product  Assertion  Analysis 


The  results  were  very  similar.  Only  the  product  size  assertions  showed  no  true  tendency. 
As  a  result,  these  two  product  assertions  were  not  included  in  SODS2. 
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4.3. 2.6  Outsourcing  Product  Related  Assertion  Analysis 

Product  goal  assertions  were  methods  set  toward  improving  outsourcing  goals 
where  product  outsourcing  strategy  was  being  practiced.  These  assertions  were  aimed  at 
the  outsourced  development  of  a  product  or  sub-product.  Figure  31  summarized  the 
product  related  type  assertions  for  the  old  data: 


Figure  31  Old  Survey  Data  Product  Related  Assertion  Analysis  [11] 

The  new  survey  data  product  related  assertion  analysis  was  listed  below  in  Figure  32: 
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Figure  32  New  Survey  Data  Product  Related  Assertion  Analysis 


The  results  were  somewhat  different.  The  new  data  showed  the  respondents  were  neutral 
on  complexity  assertion,  but  the  modularity  assertion  remained  the  same.  As  a  result,  the 
complexity  assertion  was  left  out  of  SODS2. 

4.3.3  Outsourcing  Goal  Importance  Analysis 

The  next  type  of  data  analyzed  was  the  importance  of  outsourcing  goals.  Each 
respondent  was  asked  for  input  regarding  the  importance  of  14  possible  outsourcing 
goals.  The  mean,  median,  standard  deviation  and  error  were  important,  but  using  these 
statistics  alone  could  misconstrue  the  true  results.  Fourteen  statistical  T-test  at  the  95% 
confidence  level  were  used  in  order  to  show  the  importance  of  these  goals.  This  testing 
method  accounted  for  the  difference  in  the  samples  distribution  as  well  as  the  mean 
value.  The  response  scale  from  this  question  ranged  from  1,  “Not  Important”,  to  5,  “Very 
Important.”  Figure  33  shows  the  values  captured  from  Hermann’s  analysis  of  the  old 
data: 
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Coal 

IniDortance 

Acquire  Expertise  not  Available  In-house 

3.61 

Reduce  Schedule  (Vendor  Faster) 

3.30 

Improve  Responsiveness  to  Organizational  Objectives 

3.29 

Improve  Responsiveness  to  Customer  Objectives 

3.21 

Add  People  (insufficient  in-house  capacitv ) 

3.15 

Improve  Product  Quality 

3.10 

Reduce  Schedule  (through  parallel  activities) 

3.06 

Improve  Control  over  Project  Management 

2.75 

Non-Core  Activities 

2.63 

Add  People  (short-term) 

2.54 

Reduce  Cost  (via  economies  of  scale) 

2.54 

Keep  Staffing  Levels  Stable 

2.48 

Risk  Sharing 

2.35 

Cash  Flow  from  Sale  of  Product  Rights 

1.49 

Figure  33  Old  Survey  Data  Outsourcing  Goal  Importance  [11] 

From  Figure  33,  each  goal  importance  was  ordered  from  those  with  the  highest 
mean  importance  to  the  lowest.  The  last  three  goals,  highlighted  at  the  bottom,  signified 


these  goals  were  significantly  less  than  neutral.  Figure  34  showed  the  values  captured 


from  statistical  analysis  of  the  new  data: 


Null  Hypothesis  -  Ho:  Sample  Mean  =  Neutral 

Alternate  Hypothesis  -  Ha:  Sample  Mean  >  Neutral  (95%  Cl) 


Goal 

Importance 

(sample 

mean) 

P-Val 
(chance 
importance 
=  neutral) 

T-Ratio 
(test  prob  of 
importance 
=  neutral) 

Z(alpha=  .05) 
(critical  point) 

T-dist  (  degrees  of 
freedom  ~  100) 

Rank 

Old 

Rank 

Reduce  Schedule  (To  Vendor  Faster) 

3.89 

~0% 

5.80 

1.69 

i 

4 

Improve  Responsiveness  to  Organizational  Objectives 

3.79 

~0°/o 

4.32 

1.69 

2 

1 

Reduce  Cost  (via  economies  of  scale) 

3.82 

.01% 

4.08 

1.69 

3 

12 

Improve  Quality  of  Software  Product 

3.77 

.05% 

3.76 

1.69 

4 

5 

Improve  Control  over  software  development 

3.79 

08% 

3.62 

1.69 

5 

8 

Improve  Responsiveness  to  Customer  Objectives 

3.63 

.6% 

2.91 

1.69 

6 

3 

Add  People  (Capacity) 

3.58 

.9% 

2.72 

1.69 

7 

6 

Reduce  Schedule  (Outsourcing  Parallel  Activities) 

3.49 

2% 

2.32 

1.69 

8 

7 

Keep  Staffing  Levels  Stable 

3.42 

5% 

2.01 

1.69 

9 

ii 

Risk  Sharing 

3.43 

6% 

1.96 

1.69 

10 

13 

Acquire  Expertise  Not  Available  In-House 

3.25 

31% 

1.02 

1.69 

11 

2 

Non  Core  Activities  (allows  focus  on  core  activities) 

3.14 

57% 

.58 

1.69 

12 

9 

Add  People  (Short  Term) 

3.05 

82% 

.23 

1.69 

13 

10 

Cash  Flow  from  Sale  of  Product  Rights 

2.33 

.83% 

-2.78 

1.69 

14 

14 

Notes: 

"White:  Those  goals  that  rejected  Ho  in  favor  of  Ha  (importance  greater  than  neutral) 

Light  Blue:  Those  goals  that  failed  to  reject  Ho  of  being  equal  to  neutral  (importance  is  considered  neutral) 
Grey:  Cash  flow  from  sale  of  product  rights  tend  to  be  significantly  not  important  (importance  less  than  neutral) 


Figure  34  New  Survey  Data  Outsourcing  Goal  Importance 

The  14  T-tests  indicated  some  significant  differences  between  the  new  and  old 


data.  The  new  data  showed  an  increase  in  the  mean  importance  with  each  goal  except  for 


acquiring  expertise.  This  could  indicate  that  more  companies  now  find  it  easier  to 
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acquire  software  expertise  than  did  their  counterparts  in  the  original  study.  Another 
interpretation  of  this  reduce  cost  difference  was  the  increased  competitive  nature  of 
today’s  software  industry.  During  the  technology  boom  of  the  late  1990’s,  acquiring 
software  expertise  drove  outsourcing  development.  With  more  software  outsourcing 
corporations  capable  of  such  development,  buyers  are  now  focusing  on  reducing  the  cost 
to  keep  up  with  this  competition. 

4.3.4  Outsourcing  Goal  Realization  Analysis 

The  next  step  was  to  analyze  the  realization  of  the  outsourcing  goals  described 
above.  Each  respondent  was  asked  for  input  regarding  the  satisfaction  or  realization  of 
those  14  goals.  Again  the  mean,  median,  standard  deviation  and  error  were  important, 
but  statistics  had  to  be  validated  with  T-tests  to  show  if  these  goals  were  meeting  the 
respondent’s  expectations.  The  T-test  guarded  against  having  bad  results  due  to  varying 
responses.  The  response  scale  from  this  question  ranged  from  1,  “Significantly  Worse 
Than  Expectations”,  to  5,  “Significantly  Better  Than  Expectations.”  The  middle  response 
of  3  would  indicate  that  software  outsourcing  was  right  on  target  so  anything  greater  than 
3  would  signify  outsourcing  success  while  those  lower  than  3  would  show  outsourcing 
failure  to  meet  those  goals.  Figure  35  showed  the  values  captured  from  the  old  data: 
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Goal 

Result 

Mean 

Percentage  of 
Positive 
Responses 

Acquire  Expertise  not  Available  In-house 

3.14 

79% 

Non-Core  Aeliv  ilies 

3.08 

83% 

Add  People  (insufficient  in-house  capacity) 

3.06 

81% 

Add  People  (short-term) 

2.93 

83% 

Keep  Staffing  Levels  Stable 

2.93 

83% 

Improve  Responsiveness  to  Organizational  Objectiv  es 

2.71 

62% 

Improve  Responsiveness  to  Customer  Objectives 

2.65 

59% 

Risk  Sharing 

2.65 

67% 

Improve  Product  Qualitv 

2.63 

66% 

Cash  Flow  from  Sale  of  Product  Rights 

2.70 

74% 

Reduce  Schedule  (through  parallel  activities) 

2.54 

43% 

Improve  Control  over  Project  Management 

2.54 

57% 

Reduce  Cost  (via  economies  of  scale) 

2.54 

57% 

Reduce  Schedule  (Vendor  Faster) 

2.42 

48% 

Figure  35  Old  Survey  Data  Outsourcing  Goal  Satisfaction  |11] 

From  Figure  35,  each  goal  satisfaction  was  ordered  from  those  with  the  highest 
mean  satisfaction  to  the  lowest.  Those  goals  highlighted  in  yellow  indicated  those  goals 
found  through  T-testing  to  be  equivalent  to  3  or  “right  on  target.”  The  last  five  goals  in 
white  signified  the  goals  not  found  to  be  equivalent  to  3  hence  not  “right  on  target.” 
Figure  36  showed  the  analysis  of  the  new  goals  satisfaction  data: 


Null  Hypothesis  -  Ho :  Sample  Mean  =  Neutral 

Alternate  Hypothesis  -  Ha:  Sample  Mean  <>  Neutral  (95%  Cl) 

Goal 

Importance 

(sample 

mean) 

Percent 

of 

positive 

responses 

P-Val 

(chance 

of 

being 
right  on 
target) 

T-Ratio 
(test  prob 
of  being 
right  on 
target) 

Z(alpha/2  = 
.025) 

critical  point 
T-dist 
(degrees  of 
freedom  ~ 

100) 

Old 

Mean 

Mean 

Diff. 

Add  People  (Capacity) 

3.21 

87% 

20% 

1.31 

2.02 

3.06 

0.15 

Improve  Quality  of  Software  Product 

3.16 

81% 

30% 

1.04 

2.02 

2.63 

0.53 

Keep  Staffing  Levels  Stable 

3.16 

86% 

31% 

1.02 

2.02 

2.93 

0.23 

Improve  Responsiveness  to  Organizational  Objectives 

3.15 

76% 

43% 

.80 

2.02 

2.71 

0.44 

Reduce  Schedule  (Outsourcing  Parallel  Activities) 

3.14 

77% 

41% 

.83 

2.02 

2.54 

0.60 

Improve  Responsiveness  to  Customer  Objectives 

3.10 

78% 

58% 

.56 

2.02 

2.65 

0.45 

Reduce  Cost  (via  economies  of  scale) 

3.07 

72% 

69% 

.40 

2.02 

2.54 

0.53 

Risk  Sharing 

3.00 

76% 

~  100% 

~0 

2.02 

2.65 

0.35 

Improve  Control  over  software  development 

2.79 

62% 

25% 

-1.18 

2.02 

2.54 

0.25 

Reduce  Schedule  (To  Vendor  Faster) 

2.78 

64% 

17% 

-1.40 

2.02 

2.42 

0.36 

Acquire  Expertise  Not  Available  In-House 

3.26 

84% 

14% 

1.50 

2.02 

3.14 

0.12 

Non  Core  Activities  (allows  focus  on  core  activities) 

3.03 

83% 

86% 

.18 

2.02 

3.08 

-0.05 

Add  People  (Short  Term) 

2.97 

76% 

86% 

-.18 

2.02 

2.93 

0.04 

Cash  Flow  from  Sale  of  Product  Rights 

2.81 

76% 

18% 

-1.36 

2.02 

2.70 

0.11 

Notes: 

White:  Indicates  those  important  goals  that  are  right  on  target 

Light  Blue  &  Grey  (*  Ranking):  Indicates  those  goals  that  lack  importance 

Figure  36  New  Survey  Data  Outsourcing  Goal  Satisfaction 
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The  new  goal  satisfaction  report  indicated  numerous  differences  between  the  new 
and  old  data.  The  new  data  showed  an  increase  in  every  goal  satisfaction  except  for 
reducing  non-core  activities.  Reducing  schedule,  improving  quality,  reducing  cost,  and 
increasing  responsiveness  to  both  customer  and  organizations  had  significant  increases 
from  0.60  to  0.44  respectively.  These  improvements  were  even  more  notable  since  those 
same  goals  were  the  top  five  goals  in  terms  of  importance.  With  such  significant 
differences  between  the  new  and  old  responses,  the  author  realized  they  represented 
different  populations  and  would  likely  produce  different  outsourcing  consequences. 

4.3.5  Outsourcing  Consequences  Analysis 


Outsourcing  consequences  captured  the  overall  outcome  of  outsourcing  given  the 
respondent’s  experience.  Figure  37  summarized  the  outsourcing  consequences  for  the 
old  data: 


Figure  37  Old  Survey  Data  Consequence  Analysis  [11] 
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The  new  survey  data  consequence  analysis  was  listed  below  in  Figure  38: 
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Figure  38  New  Survey  Data  Consequence  Analysis 

The  results  were  very  different.  Every  consequence  had  better  present  results.  While 


these  two  figures  provided  an  excellent  overview  of  the  changes,  Figure  39  pointed  out 


the  actual  differences: 
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Consequence 

Response 

Mean 

Stand. 

Dev 

Stand. 

Error 

Worse 

% 

Neutral 

% 

Better 

% 

Old 

worse  % 

Old 

better  % 

Diff. 

better 

Diff 

worse 

Schedule  Flexibility 

4.50 

1.60 

0.24 

0.28 

0.15 

0.57 

0.26 

0.55 

0.02 

0.02 

Product  Quality 

4.86 

1.23 

0.19 

0.18 

0.13 

0.69 

0.27 

0.47 

0.22 

-0.09 

Responsiveness  to  Organizational  Objectives 

4.52 

1.30 

0.20 

0.16 

0.30 

0.54 

0.31 

0.42 

0.12 

-0.15 

Responsiveness  to  Customer 

4.73 

1.21 

0.18 

0.10 

0.34 

0.56 

0.29 

0.36 

0.20 

-0.19 

Cost  Associated  with  Change 

3.73 

1.59 

0.24 

0.38 

0.18 

0.44 

0.35 

0.33 

0.11 

0.03 

Rework 

4.16 

1.41 

0.22 

0.37 

0.35 

0.28 

0.34 

0.31 

-0.03 

0.03 

Risks 

3.75 

1.54 

0.23 

0.34 

0.23 

0.43 

0.39 

0.31 

0.12 

-0.05 

Level  of  Inhouse,  Non- Core  Activities 

3.36 

1.67 

0.25 

0.25 

0.13 

0.62 

0.22 

0.29 

0.33 

0.03 

Costs 

2.98 

1.65 

0.24 

0.21 

0.07 

0.72 

0.46 

0.29 

0.43 

-0.25 

Schedule  Duration 

3.41 

1.33 

0.20 

0.24 

0.15 

0.61 

0.50 

0.28 

0.33 

-0.26 

Likelihood  of  Project  Failure 

3.86 

1.46 

0.22 

0.33 

0.31 

0.36 

0.31 

0.26 

0.10 

0.02 

Control  over  Final  Product 

4.27 

1.28 

0.19 

0.27 

0.34 

0.39 

0.38 

0.22 

0.17 

-0.11 

Control  over  Development  Process 

4.00 

1.56 

0.23 

0.43 

0.20 

0.37 

0.48 

0.23 

0.14 

-0.05 

Administrative  Overhead 

3.61 

1.77 

0.26 

0.37 

0.11 

0.52 

0.44 

0.22 

0.30 

-0.07 

Project  Learning  Curve 

3.93 

1.51 

0.23 

0.33 

0.27 

0.40 

0.31 

0.20 

0.20 

0.02 

Intellectual  Capital 

4.41 

1.22 

0.18 

0.12 

0.54 

0.34 

0.13 

0.19 

0.15 

-0.01 

Visibility  in  Development  Process 

4.14 

1.34 

0.21 

0.32 

0.33 

0.35 

0.51 

0.17 

0.18 

-0.19 

Turf  Ware 

3.93 

1.44 

0.22 

0.26 

0.38 

0.36 

0.39 

0.08 

0.28 

-0.13 

Inhouse  Personnel  Turnover 

4.02 

1.26 

0.19 

0.23 

0.56 

0.21 

0.09 

0.06 

0.15 

0.14 

Culture,  Language  or  Location  Problems 

3.98 

1.14 

0.17 

0.28 

0.47 

0.25 

0.37 

0.03 

0.22 

-0.09 

Figure  39  New  Survey  Data  Consequence  Differences  Analysis 
The  differences  (new  minus  old)  were  shown  in  the  last  two  columns.  Every 
consequence,  except  schedule  flexibility  and  rework,  had  a  10  to  40  percent  increase  in 
success.  The  negative  responses  remained  either  constant  or  showed  a  slight  decline. 
Cost,  product  quality,  schedule  duration,  responsiveness  to  both  customer  and 
organizations,  and  level  of  non  core  activities  showed  to  have  the  most  positive  shifts, 
now  proving  outsourcing  could  produce  positive  consequences  in  each  of  these  areas. 
Five  of  the  six  consequences  followed  the  previously  discussed  results  from  the  goal 
importance  and  realization.  Having  this  consistency  within  the  survey  was  another 
harbinger  of  good  survey  data  quality,  thus  should  strengthen  the  support  for  SODS2. 


4.3.6  Survey  Data  Analysis  Summary 


The  outsourcing  experience,  assertions,  goal  importance  and  realization,  and 
consequences  old  and  new  survey  results  were  analyzed  and  compared  with  each  other. 
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Significant  differences  were  discovered  throughout  the  analysis.  The  old  and  new  survey 
results  appeared  to  not  be  from  the  same  population  as  thought  to  be  true  during 
methodology  conception.  Further  explanation  of  such  differences  required  regression 
modeling. 

4.4  Regression  Model  Implementation 

This  implementation  phase  had  three  goals  resulting  from  the  previous  analyses: 

1.  Validate  regression  modeling  techniques  are  similar  to  those  same  techniques  used  in 
building  the  models  for  SODS1, 

2.  Research  differences  between  the  new  and  used  data  populations,  and 

3.  Build  regression  models  for  outsourcing  consequences  similar  to  those  used  in 
SODS1  with  the  exception  of  using  all  three,  old,  new  and  used,  data  sets  with  75% 
stepwise  entry/removal  statistics. 

4.4.1  Regression  Modeling  Technique  Validation 

Before  any  comparisons  could  be  made  to  the  old  survey  data  goal  realization 
regression  models,  methodology  was  undertaken  to  validate  the  old  model  making 
techniques  were  the  same  as  the  new  model  making  techniques  used  in  this  research.  The 
same  stepwise  entry  statistic  of  95%  and  removal  statistic  of  90%  were  used.  Goal 
realization  of  response  to  organization  was  randomly  picked  for  this  validation  effort. 
Appendix  A  showed  an  exact  match  between  this  model  and  the  model  documented  in 
the  Hermann’s  dissertation.  To  reconfirm  this  result,  response  to  customer  goal 
realization  was  picked,  and  as  shown  in  Appendix  A,  another  exact  match  was  produced. 
This  confirmed  that  this  author’s  modeling  techniques  were  consistent  with  the 
techniques  used  in  the  dissertation. 
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4.4.2  Regression  Model  Differences  in  14  Goal  Realizations 

Once  the  modeling  techniques  were  validated,  14  goal  realizations  models  were 
created  using  the  new  survey  data  and  the  same  entry/removal  statistics,  95%/90%,  that 
were  used  by  Hermann  in  creating  his  14  goal  realization  models.  The  purpose  for  the 
creation  of  these  models  was  two  fold: 

1)  Help  explain  the  differences  between  the  new  and  used  survey  data  and 

2)  Provide  an  overview  of  the  modeling  techniques  that  was  going  to  be  used 
in  the  creation  of  the  consequence  models. 


Table  34  showed  the  differences  between  the  models: 


Table  34  Differences  in  Old  and  New  Goal  Realization  Models 


New  Goal  Realization  Model 

Old  Goal  Realization  Model 

AddPeopleCapacity  =  3.10  -  (1.97)sys- 
embed  -  (1.16)ent-acctng  +  (,75)proc- 
maint 

[No  Model  Found  Matching  Entry/Removal 
Stats] 

AddPeopleShortTenn  =  2.97  -  (.97)sys- 
device  +  (2.02)comp-os 

AddPeopleShortTenn  =  3.03  -  (,62)prod- 
commoncust  +  (.87)ent-acctng 

CashFlow  =  2.74  -  (.88)comp- 
development  +  (,74)proc-toolsup 

CashFlow  =  2.88  -  (1.02)sys-device  - 
(.43)ent-web 

Control  =  2.57  +  (2.43)comp-case  + 
(1.43)comp-os 

Control  =  2.55  +  (.99)proc-reeng  -  (,69)proc- 
req 

Expertise  =  3.71  -  (.71)prod-custom 

Expertise  =  2.86  +  (.59)sys-all  - 
( 1 .20)shrink-internet  +  (.64)proc-reeng 

NonCore  =  2.88  -  (1.09)shrink-util  + 

( 1 . 14)comp-domain  +  (1 .48)ent-mnft 

NonCore  =  3.25  -  (.85)sys-device  - 
(.48)prod-cots 

Quality  =  3.28  +  (.86)proc-sweng  - 
(.63)prod-cust 

Quality  =  2.73  -  (1.33)sys-device 

ReduceCost  =  2.88  +  (.82)proc-reeng 

ReduceCost  =  2.66  -  (.58)proc-fielding 

ScheduleParallel  =  2.76  +  (1.1  l)proc- 
spec 

ScheduleParallel  =  2.70  -  (.66)proc-sweng  + 
(1.05)sys-avionics 

Schelule Vendor  =  2.68  -  (1.50)proc- 
design  +  (.97)proc-test  +  (1.21)proc-spec 

ScheluleVendor  =  2.25  +  (.57)proc-req 

ResponseCustomer  =  2.73  +  (1.34)comp- 
os  -  (1.05)comp-development  - 
( 1 .20)proc-design  +  ( 1 .36)proc-test  + 
(.64)proc-applsup  +  ( 1 . 1 8)prod-none 

ResponseCustomer  =  2.92  -  (.85)proc-sweng 
-  (l.Ol)sys-device  +  (1.51)ent-oes  +  (.97)sys- 
avionics 
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ResponseOrganization  =  2.97  + 
(1.36)shrink-util  +  (1.53)comp-os 

ResponseOrganization  =  2.31  -(1.1 8)sys- 
device  -  (.61)proc-sweng  +  (.79)proc-coding 

RiskSharing  =  2.90  -  (,73)comp- 
development  +  (1.60)ent-payroll  + 
(l.lO)proc-none 

RiskSharing  =  2.86  -  (.53)proc-sweng 

StaffStable  =  3.03  =  (1.98)comp-os 

StaffStable  =  2.78  +  (.69)proc-maint  - 
(,76)proc-cm  +  (,45)proc-reeng  +  (1.22)prod- 
none 

The  differences  were  significant.  There  were  no  common  input  variables  between  the  old 
and  new  models.  Only  four  of  the  14  had  a  similar  intercept  +  10%  which  could  have 
been  forecasted  given  the  goal  satisfaction  analysis  in  the  above  paragraphs.  After  this 
point,  the  research  methodology  was  modified  slightly  due  to  these  differences  uncovered 
in  these  models  and  during  the  analysis.  Adding  the  new  and  used  data  together  might 
overshadow  important  developments  in  software  outsourcing.  Yet,  if  the  data  was  not 
added,  the  similarities  running  parallel  through  both  data  sets  would  never  be  manifested. 
The  solution  to  this  dilemma  was  unavoidable,  all  three  (new,  old  and  combined)  data 
models  were  created  and  used  in  SODS2  to  provide  the  support  required  to  help  decision 
makers. 

Another  concern  brought  forth  from  these  changes  was  the  number  of  variables 
included  in  each  model  under  the  stepwise  entry  statistic  of  95%.  This  tight  restriction 
failed  to  find  any  input  variables  for  the  adding  people  for  capacity  consequence  model. 
Evaluating  the  34  goal  satisfaction  and  consequence  models  presented  in  Hermann’s 
dissertation,  the  author  found  that  no  input  variables  had  been  removed.  This  was  due  to 
the  tightly  constrained  removal  statistic  at  90%.  Given  this  and  the  fact  that  SODS2 
relied  on  the  accuracy  of  the  NN  learning  mechanism,  the  entry/removal  statistics 
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expanded  to  75%  so  that  more  input  variables  between  the  three  data  sets  could  be 
researched.  This  change  resulted  in  an  additional  step  in  the  research.  The  old  data 
models  were  going  to  have  to  be  recreated  along  with  the  new  and  combined  models 
using  the  new  75%  entry/removal  statistics. 

4.4.3  SODS2  Consequence  Regression  Models 

While  the  model  changes  caused  an  increase  in  effort  for  this  phase,  this  additional 
work  was  crucial  to  assuring  that  all  of  the  models’  input  variables  were  being  caught 
between  the  new,  old  and  combined  models.  In  Appendices  B,  C  and  D,  the  new,  old  and 
combined  consequence  regression  models  were  listed  in  their  fullest  detail.  This  section 
explained  the  differences  between  the  old  and  new  outsourcing  consequence  models  as 
summarized  in  Figure  40.  The  model  differences  showed  that  the  project  data  samples 
appeared  to  be  from  different  populations.  This  meant,  according  to  the  survey  data,  that 
the  software  outsourcing  world  had  significantly  changed  from  the  original  data 
collection  of  projects  from  1995-2000  to  the  newer  data  collection  of  projects  from  2000- 
2003.  The  differences  may  have  been  attributed  to  dramatic  changes  in  the  technology, 
economy  and  a  longer  history  of  software  outsourcing  experiences.  In  some  cases,  the 
intercept  values  of  each  model  were  significantly  different  which  would  clearly 
demonstrate  this  concept.  While  each  independent  (predictor)  variable  was  compared 
across  the  models,  the  reader  is  cautioned  to  remember  that  a  significant  change  in  the 
intercept  could  intensify  or  eliminate  differences  between  each  predictor  variable. 
Similar  caution  should  be  paid  to  interpolation  effects.  Interpolation  was  the  effect  of  the 
model  trying  to  make  predicted  outcomes  based  on  little  or  no  matching  patterns  in  the 
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survey.  The  new  data  samples  found  little  data  relating  to  system  (avionics,  embedded 
and  development  tool)  software  domains,  shrinkwrap  (entertainment  and  utility)  software 
domains,  component  (CASE  and  library  class)  software  domains,  enterprise 
(manufacturing,  order  entry,  scripting  and  website)  software  domains,  and  outsourced 
(documentation  and  none)  processes.  Similarly,  the  old  data  samples  found  little  data 
relating  to  shrinkwrap  (entertainment  and  utility)  software  domains,  component  (CASE 
and  library  class)  software  domains,  enterprise  (manufacturing,  payroll  and  order  entry) 
software  domains,  outsourced  (none)  process  and  outsourced  (none)  products.  These 
independent  variables  will  thus  be  skewed  more  toward  the  intercept  as  the  result  of 
having  no  observations  or  toward  a  biased  mean  as  the  results  of  having  few  samples  to 
be  used  in  the  computations.  This  was  also  another  reason  why  the  combined  models 
were  created  even  though  there  were  many  differences  between  the  two  data  populations. 
All  models  along  with  their  perfonnance  measures  were  included  in  Figure  40  listed  on 
the  next  four  pages. 
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Figure  40  Summary  of  Old,  New  and  Combined  Models 
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(Figure  40  cont.)  Summary  of  Old,  New  and  Combined  Models 
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(Figure  40  cont.)  Summary  of  Old,  New  and  Combined  Models 
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(Figure  40  cont.)  Summary  of  Old,  New  and  Combined  Models 
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4.4.3. 1  Cost  Consequence 

The  cost  consequence  dictated  the  money  saved  for  projects  using  outsourcing 
rather  than  in-house  development.  Even  though  many  organizations  sought  using 
outsourcing  to  reduce  the  overall  cost  of  the  software  development  effort,  figures  showed 
this  goal  was  not  outsourcing’s  strongest  suit.  Both  models  started  out  with  similar 
intercepts  that  held  this  consequence  at  neutral  which  was  even  with  those  organizations 
that  chose  not  to  outsource.  Embedded  systems,  operating  system  components,  and 
enterprise  (accounting  and  order  entry)  software  domains  along  with  outsourcing 
(specifications  and  configuration  management)  processes  and  COTS  products  were 
included  in  the  original  model  but  not  in  the  new  model.  The  new  model  found 
enterprise  payroll  software  domain  and  outsourcing  customizable  common  software 
products  as  indicating  variables  which  were  not  included  in  the  old  model.  System 
communication,  CASE  component  and  enterprise  manufacturing  software  domains  found 
a  common  theme  in  both  models.  System  communication  software  domain  saw  an  old 
model  decreasing  effect  change  into  an  increasing  effect  for  the  new  model.  CASE 
component  software  domain  correlated  with  a  significant  decrease  in  cost  and  with 
similar  magnitudes  between  the  two  models.  Enterprise  manufacturing  software  domain 
associated  with  higher  cost  though  the  magnitude  of  this  increase  was  dramatically  less  in 
the  new  model.  Outsourcing  customizable/  specialized  software  products  showed  an 
increase  in  cost  from  the  old  model  to  a  decrease  in  cost  from  the  new  model.  Both 
models  were  considered  somewhat  similar  with  only  17  marks  separating  the  two.  The 
older  model  did  have  higher  accuracy  with  a  0.44  R  value  and  less  noise  with  a  1.17 
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Regression  MSE  (RMSE)  value  making  it  the  choice  model  to  use  despite  being  built 
with  older  data. 

4.4.3.2  Schedule  Duration  Consequence 

The  schedule  duration  consequence  defined  the  outsourcing  goals  and  efforts  used 
in  order  to  complete  the  software  development  project  faster.  Unlike  the  above  cost 
consequence,  both  intercepts  from  both  models  signified  that  outsourcing  was  meeting 
this  goal  somewhat  by  slightly  decreasing  the  project  schedule  duration  when  compared 
to  those  not  outsourcing.  The  indicating  variables  included  in  the  old  model  and  not  in 
the  new  model  included:  (avionic  and  communication)  system  software  domains, 
enterprise  manufacturing  software  domain,  outsourcing  (design,  documentation,  fielding 
and  configuration  management)  processes,  and  outsourcing  custom/special  products. 
This  moved  these  variables  in  line  with  the  intercept  of  having  a  slightly  decrease 
schedule.  The  (embedded  and  device)  system  software  domains,  shrink-wrap  utility 
software  domain,  enterprise  scripting  software  domain,  and  outsourcing  (maintenance 
and  training)  processes  found  to  be  included  in  the  new  model  but  not  in  the  original 
model.  Shrink-wrap  business  software  domain,  enterprise  accounting  software  domain, 
and  outsourcing  reengineering  process  were  common  to  both  models  with  the  enterprise 
accounting  software  domain  and  outsourcing  reengineering  process  sharing  almost  the 
exact  multiplier.  The  shrink-wrap  business  software  domain  showed  to  have  an  increase 
effect  to  schedule  whereas  in  the  original  model  it  was  listed  to  having  a  negative  effect. 
Given  these  changes,  the  two  models  were  somewhat  similar  with  only  21  marks 
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2 

separating  them.  The  newer  model  did  have  higher  accuracy  with  a  R  value  of  0.5 1  and 
less  noise  with  a  RMSE  value  of  1.04  thus  making  it  the  choice  model  to  use. 

4.4.3.3  Intellect  Capital  Consequence 

The  intellectual  capital  consequence  referred  to  an  organization’s  (legal) 
ownership  rights  of  a  developed  software  product.  Higher  level  rights  were  indicative  of 
stronger  ownership.  A  loss  of  intellectual  capital  was  a  concern  for  organizations, 
causing  many  to  forgo  outsourcing  key  software  development  for  fear  that  external 
companies  would  hold  their  organization  “hostage”  since  they  alone  possessed  key 
information.  The  initial  model  showed  an  intercept  with  a  slight  increase  in  intellectual 
capital  rights  as  compared  to  their  counterparts  who  do  not  outsource  software 
development.  The  new  model  showed  a  similar  sized  decrease  in  intellectual  capital 
rights  according  to  the  intercept.  The  difference  of  1.33  indicated  that  the  new  sample  of 
outsourcing  project  was  experiencing  significantly  reduce  intellectual  capital 
consequences.  Shrink-wrap  (business  and  utilities)  projects,  (CASE  and  development 
tools)  component  projects,  enterprise  (manufacturing  and  scripting)  software  projects, 
outsourced  (reengineering,  application  support,  training  and  coding/implementation) 
processes,  and  outsourced  (custom/specialized)  products  were  included  in  the  initial 
model,  but  were  no  longer  significantly  correlated  with  intellectual  capital  consequences. 
Thus  these  project  types  and  processes  no  longer  experienced  significant  differences  with 
their  in-house  counterparts.  Several  new  independent  variables  including  system 
development  software,  shrink-wrap  internet  products,  operating  system  components, 
enterprise  accounting  software,  outsourced  (maintenance,  configuration  management  and 
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tool  support)  processes  were  included  in  the  new  model  and  were  absent  in  the  original 
model.  Operating  system  projects  and  configuration  management  outsourcing  were  now 
correlated  with  significant  reduced  intellectual  capital  while  tool  support  outsourcing  and 
Internet  shrink-wrap  products  among  the  sample  projects  were  indicative  of  significant 
higher  intellectual  capital.  Domain  framework  components  and  outsourcing  (fielding  and 
software  engineer  support)  processes  were  common  to  both  models.  While  outsourcing 
the  fielding  process  was  still  indicative  of  reduced  intellectual  capital,  the  magnitude  of 
the  reduction  was  significantly  less  than  the  original  model.  In  both  models,  domain 
framework  components  indicated  higher  intellectual  capital.  In  the  original  model,  the 
outsourcing  software  engineer  support  was  associated  with  a  lower  intellectual  capital. 
The  new  model  experienced  a  complete  reverse  effect  showing  software  engineering 
support  correlated  with  a  significant  higher  amount  of  intellectual  capital.  Both  models 
shared  a  higher  than  normal  R  values  along  with  lower  than  normal  noise  (RMSE) 
values.  The  new  model  was  the  model  of  choice  due  to  being  built  with  this  latest 
observed  data. 

4.4.3.4  Schedule  Flexibility  Consequence 

Schedule  flexibility  consequence  gave  the  outsourcing  organization  the  ability  to 
change  their  schedule  for  possible  reasons  such  as:  increased  number  of  projects  being 
developed,  decreased  number  of  projects  being  developed,  handle  an  increase  in 
maintenance  or  reengineering  tasks,  or  adding  skilled  personnel  to  any  undennanned 
position  that  would  otherwise  cause  a  bottleneck  in  the  project  schedule.  There  was  a 
slight  difference  between  the  original  and  new  model.  The  original  started  out  with  a 
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neutral  outcome  meaning  it  was  exactly  the  same  as  those  who  decided  not  to  outsource. 
The  new  model,  however,  showed  to  have  a  slightly  increase  amount  of  schedule 
flexibility  due  to  outsourcing.  System  (avionics  and  communications)  software  domains, 
operating  system  component  software  domain,  and  outsourcing  (requirements, 
reengineering,  specification  and  fielding)  processes  were  found  in  the  original  model  but 
not  in  the  new  model.  Embedded  system  software  domain,  shrink-wrap  business 
software  domain,  component  (class  library  and  development  tool)  software  domains, 
enterprise  payroll  software  domain,  outsourcing  (design,  testing,  coding,  configuration 
management  and  software  engineer  support)  processes  and  outsourcing  customizable 
common  products  were  all  indicator  variables  included  in  the  new  model  but  not  the  old 
model.  Enterprise  accounting  and  CASE  component  software  domains  were  common  in 
both  models.  Enterprise  accounting  had  almost  the  same  multipliers,  but  CASE 
component  software  domain  found  to  have  twice  the  negative  effect  from  the  old  model 
to  the  new  model.  The  two  models  were  somewhat  separated  with  31  marks  between 
them.  The  newer  model  did  have  higher  accuracy  with  a  R  value  of  0.74  and  less  noise 
with  a  RMSE  value  of  0.97,  thus  making  it  the  choice  model  to  use. 

4.4.3. 5  Admin  Overhead  Consequence 

Administration  overhead  consequence  concerned  itself  with  goals  to  reduce  the 
amount  of  management,  legal  support  and  paperwork  needed  by  a  software  development 
project.  It  was  assumed  by  many  organizations  that  the  overhead  involved  in  outsourcing 
was  a  prime  reason  for  not  outsourcing.  Figures  have  shown  quite  the  opposite,  and  in 
some  cases,  actually  decreased  the  amount  of  administrative  overhead.  Both  models 


106 


AFIT/GCS/ENG/04-16 


started  with  an  intercept  placing  this  consequence  as  equal  to  those  organizations  not 
outsourcing.  Those  indicating  variables  found  in  the  original  model  but  not  in  the  new 
model  include:  system  avionics  software  domain,  shrinkwrap  utilities  software  domain, 
domain  framework  component  software  domain,  enterprise  (manufacturing  and  order 
entry)  software  domains,  and  outsourcing  (application  support,  documentation,  fielding 
and  configuration  management)  processes.  Those  indicating  variables  found  in  the  new 
model  but  not  in  the  original  include:  component  (class  library  and  operating  system) 
software  domains,  outsourcing  (requirements,  testing,  tool  support  and  software  engineer 
support),  and  outsourcing  no  products.  Shrinkwrap  internet  software  domain  and 
outsourcing  customizable  COTS  products  were  common  between  both  models  with 
approximately  the  same  increasing  magnitude.  The  two  models  were  somewhat  different 
with  28  marks  separating  them.  The  newer  model  did  have  higher  accuracy  with  a  R~ 
value  of  0.53,  but  the  old  model  proved  to  have  less  noise  with  a  RMSE  value  of  1.05. 
The  new  model  was  the  model  of  choice  due  to  the  fact  it  was  slightly  more  accurate  and 
built  with  the  latest  data. 

4.4.3. 6  Control  Process  Consequence 

Controlling  the  development  process  was  a  goal  for  organizations  dissatisfied 
with  their  current  control  of  their  own  in-house  development  shops.  The  initial  intercept 
showed  a  small  difference  between  the  original  and  new  model.  The  original  model 
found  that  outsourcing  could  slightly  improve  this  consequence  whereas  the  new  model 
showed  only  a  neutral  outcome.  In  either  case,  outsourcing  did  no  worse  at  this  control 
than  those  organizations  not  outsourcing.  Domain  frameworks  component  software 
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domain,  outsourcing  (design,  reengineering,  specification  and  tool  support)  processes, 
and  outsourcing  customizable  COTS  products  were  indicating  variable  in  the  original 
model  but  not  in  the  new  model.  System  (avionic  and  device)  software  domains, 
operating  system  component  software  domain,  enterprise  (manufacturing  and  payroll) 
software  domains,  outsourcing  (application  support,  coding  and  software  engineer 
support)  processes,  and  total  process  outsourcing  were  indicating  variables  found  in  the 
new  model  but  not  in  the  original  model.  Shrinkwrap  utility  software  domain, 
development  tools  component  software  domain,  and  outsourcing  COTS  products  were 
common  indicating  variables  between  both  models.  Development  tools  component 
software  domain  and  outsourcing  COTS  products  found  to  alternate  from  decreasing  this 
consequence  in  the  old  model  to  increasing  this  consequence  in  the  new  model. 
Shrinkwrap  utility  software  domain  saw  the  reverse  of  the  above  with  it  correlating  with 
an  increase  of  this  consequence  in  the  old  model.  While  in  the  new  model,  this  correlated 
with  a  decrease  in  process  control.  The  two  models  were  somewhat  different  with  25 
marks  separating  them.  The  newer  model  did  have  higher  accuracy  with  a  R2  value  of 
0.63  and  less  noise  with  a  RMSE  value  of  1 . 10  thus  making  it  the  choice  model  to  use. 

4.4.3. 7  In-house  Non  Core  Consequence 

Effort  spent  on  in-house  non-core  activities  should  be  kept  to  a  minimum.  Some 
organizations  sought  to  use  outsourcing  to  minimize  such  non-core  in-house  activities. 
This  saved  time  would  allow  organizations  to  put  more  in-house  effort  towards  core  or 
strategic  type  business.  The  intercept  from  the  old  model  showed  a  neutral  outcome 
which  would  be  equal  to  those  organizations  not  outsourcing.  The  new  model  found  a 
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more  favorable  position.  It  had  a  slight  decrease  of  the  effort  spent  on  non-core 
activities.  These  intercepts  were  suspected  to  change  according  to  the  selected  indicating 
variable  being  used.  For  example,  those  organizations  seeking  to  outsource  their  design 
process  tended  to  find  an  increase  in  time  spent  on  non-core  tasks,  but  outsourcing 
requirements  process  would  significantly  decrease  time  used  on  non-core  activities. 
System  avionic  software  domain,  enterprise  (accounting,  manufacturing,  order  entry  and 
web-site)  software  domains,  outsourcing  (tool  support  and  software  engineer  support) 
processes,  outsourcing  COTS  product,  and  total  process  outsourcing  showed  to  be 
indicators  in  the  original  models  but  not  in  the  new  models.  System  communication 
software  domain,  operating  system  component  software  domain,  enterprise  scripting 
software  domain,  and  outsourcing  (requirements,  design,  testing  and  reengineering) 
processes  saw  their  way  into  the  new  model  but  not  the  old  model.  Development  tools 
component  software  domain  found  to  be  the  only  common  indicating  variable  between 
both  models.  It  shared  an  increasing  effect  on  this  consequence.  The  two  models  were 
somewhat  similar  with  22  marks  separating  them.  The  newer  model  did  have  higher 
accuracy  with  a  R  value  of  0.44,  but  the  old  model  had  less  noise  with  a  RMSE  value  of 
1.07.  The  new  model  was  the  model  of  choice  due  to  the  fact  it  was  a  slightly  more 
accurate  and  built  with  the  latest  data. 

4.4.3.8  In-house  Personnel  Turnover  Consequence 

Many  organizations  feared  that  outsourcing  would  increase  in-house  personnel 
turnover.  Those  that  have  outsourced  found  in  many  cases  this  was  not  true. 
Outsourcing  seemed  to  keep  job  conditions  favorable  so  that  in-house  personnel  did  not 
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leave  the  organization.  There  was  a  slight  intercept  difference  between  the  original  and 
new  model.  The  original  started  out  with  a  neutral  outcome  meaning  it  was  exactly  the 
same  as  those  who  decided  not  to  outsource.  The  new  model  however  showed  favorable 
signs  to  slightly  and  maybe  even  significantly  decreased  turnover.  System  (avionics  and 
device)  software  domains,  shrinkwrap  business  software  domain,  CASE  component 
software  domain,  enterprise  (accounting  and  order  entry)  software  domains,  and 
outsourcing  (maintenance,  configuration  management  and  software  engineer  support) 
processes  were  found  in  the  original  model  but  not  in  the  new  model.  Component 
(operating  system  and  development  tool)  software  domains  and  outsourcing 
(requirements,  testing,  reengineering  and  specification)  processes  were  all  indicator 
variables  included  in  the  new  model  but  not  the  old  model.  Enterprise  scripting  software 
domain  and  outsourcing  design  process  were  common  in  both  models.  Enterprise 
scripting  software  domain  had  the  same  increasing  effect  but  showed  a  higher  magnitude 
of  this  effect  in  the  new  model.  Outsourcing  design  process  showed  to  have  a  slightly 
negative  effect  in  the  original  model  but  a  significant  increasing  effect  in  the  new  model. 
The  two  models  were  somewhat  similar  with  18  marks  separating  them.  The  newer 
model  did  have  higher  accuracy  with  a  R  value  of  0.51.  The  older  model  had  less  noise 
with  a  RMSE  value  of  0.81.  The  new  model  was  the  model  of  choice  due  to  the  fact  it 
was  more  accurate  and  built  with  the  latest  data. 

4.4.3.9  Learning  Curve  Consequence 

Learning  curve  consequence  concerned  itself  with  goals  to  reduce  the  learning 
curve.  Time  spent  by  employees  climbing  up  the  learning  curve  in  order  to  develop  the 
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software  was  expensive  to  both  the  schedule  and  to  the  total  economical  price  tag 
associated  with  creating  the  software.  For  each  hour  spent  learning  what  must  be  learned, 
the  cost  of  the  software  project  rose.  For  these  reasons,  some  organization  found  it 
important  to  keep  the  learning  curve  as  small  as  possible.  Outsourcing  was  found  to 
provide  organizations  a  solution  to  reducing  this  learning  curve.  The  old  model  started 
with  its  intercept  placing  this  consequence  as  equal  to  those  organizations  not 
outsourcing.  The  new  model’s  intercept  showed  favorable  signs  of  slightly  reducing  this 
learning  curve  thus  providing  organizations  a  solution  not  realized  by  those  who  simply 
do  not  outsource.  Those  indicating  variables  found  in  the  original  model  but  not  in  the 
new  model  included:  enterprise  (scripting  and  order  entry)  software  domains  and 
outsourcing  maintenance  process.  Those  indicating  variables  found  in  the  new  model  but 
not  in  the  original  included:  shrinkwrap  utility  software  domain,  enterprise  payroll 
software  domain,  outsourcing  (application  support,  training,  coding  and  configuration 
management)  processes,  and  total  product  outsourcing.  Component  (CASE  and  class 
library)  software  domains  and  outsourcing  customizable  common  products  were  common 
between  both  models.  Component  (CASE  and  class  library)  software  domains 
significantly  decreased  the  consequence  in  both  models.  Outsourcing  customizable 
common  products  saw  an  increase  in  both  models  (a  significant  increase  in  the  newer 
model).  The  two  models  were  somewhat  similar  with  14  marks  separating  them.  The 
newer  model  did  have  higher  accuracy  with  a  R  value  of  0.59,  but  the  old  model  proved 
to  have  a  little  less  noise  with  a  RMSE  value  of  1.06.  The  new  model  was  the  model  of 
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choice  due  to  the  fact  it  did  a  much  better  job  explaining  the  outcome  and  built  with  the 
latest  data. 

4.4.3.10  Risk  Consequence 

Risk  consequence  referred  to  the  ability  for  a  project  to  use  outsourcing  in 
minimizing  risks.  Risk  mitigation  efforts  among  contract  organizations  provided  the 
ability  to  pool  creative  solutions  from  multiple  sources.  The  intercept  from  the  old  model 
showed  a  neutral  to  slightly  increased  outcome  which  would  be  at  least  equal  to  those 
organizations  not  outsourcing.  The  new  model  showed  a  complete  different  picture.  It 
showed  that  risk  minimization  through  outsourcing  was  slightly  decreased  which  may  be 
raised  significantly  depending  upon  how  one  chooses  to  outsource.  System 
communication  software  domain,  enterprise  (order  entry,  scripting  and  web-site)  software 
domains,  outsourcing  (configuration  management  and  software  engineer  support) 
processes,  and  outsourcing  COTS  product  proved  to  be  indicators  in  the  original  models 
but  not  in  the  new  models.  System  embedded  software  domain,  shrinkwrap  utility 
software  domain,  operating  system  component  software  domain,  enterprise  payroll 
software  domain,  and  outsourcing  customizable  common  products  saw  their  way  into  the 
new  model  but  not  in  the  old  model.  Surprisingly,  there  were  no  common  indicating 
variables.  The  two  models  were  similar  with  15  marks  separating  them.  The  newer 
model  did  have  higher  accuracy  with  a  R  value  of  0.38  and  proved  to  have  less  noise 
with  a  RMSE  value  of  1.28.  The  new  model  was  the  model  of  choice  due  to  the  fact  it 
did  a  little  better  job  explaining  the  outcome  and  was  built  with  the  latest  data. 
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4.4.3.11  Quality  Consequence 

The  quality  consequence  emphasized  the  quality  of  the  end  product.  This  end 
product  could  have  been  the  software,  a  support  process  or  documents.  Both  models 
started  out  with  similar  intercepts  that  hold  this  consequence  between  slightly  increasing 
to  a  neutral  outcome  which  was  at  least  the  same  level  of  quality  of  those  who  do  not 
outsource.  Outsourcing  (reengineering  and  software  engineering  support)  processes  and 
outsourcing  COTS  products  were  included  in  the  original  model  but  not  in  the  new 
model.  The  new  model  found  system  (embedded  and  device)  software  domains, 
shrinkwrap  (utility  and  internet)  software  domains,  component  (CASE  and  class  library) 
software  domains,  outsourcing  (specification  and  coding)  processes,  outsourcing 
custom/specialized  software  products,  and  total  process  outsourcing  as  indicating 
variables  which  were  not  included  in  the  old  model.  Again,  no  indicating  variables  were 
found  in  common.  These  two  models  were  similar  with  only  1 5  marks  separating  them, 
but  the  combined  model  suffered  a  poor  0.23  R  .  The  newer  model  achieved  higher 
accuracy  with  a  R  value  of  0.48  and  less  noise  with  a  RMSE  value  of  1.01  thus  making  it 
the  choice  model  to  use. 

4.4.3.12  Rework  Consequence 

The  last  thing  that  organization  desired  was  rework  especially  after  paying  an 
outsourced  organization  money  for  products  and  processes  which  they  found  themselves 
having  to  redo.  The  intercept  from  both  models  showed  a  neutral  rework  outcome  which 
would  be  at  least  equal  to  the  amount  of  rework  accomplished  by  organization  not 
outsourcing.  System  (communication  and  device)  software  domains,  shrinkwrap  internet 
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software  domain,  enterprise  scripting  software  domain,  outsourcing  reengineering 
process,  and  outsourcing  COTS  products  proved  to  be  indicators  in  the  original  models 
but  not  in  the  new  models.  System  avionics  software  domain,  shrinkwrap  utility  software 
domain,  component  development  tool  software  domain,  outsourcing  application  support 
process,  and  total  product  outsourcing  saw  their  way  into  the  new  model  but  not  in  the 
old  model.  Outsourcing  software  engineer  support  process  was  found  to  be  the  only 
indicating  variable  common  in  both  models.  Outsourcing  software  engineer  support 
process  saw  in  the  original  model  to  increase  this  rework,  but  in  the  new  model,  this 
indicating  variable  was  found  correlated  with  a  decrease  of  rework.  The  two  models 
were  similar  with  14  marks  separating  them.  The  newer  model  did  have  higher  accuracy 
with  a  R~  value  of  0.32  and  proved  to  have  similar  noise  with  an  approximate  RMSE 
value  of  1.20.  The  new  model  was  the  model  of  choice  due  to  the  fact  it  did  a  better  job 
explaining  the  outcome  and  was  built  with  the  latest  data. 

4.4.3.13  Visibility  Consequence 

Visibility  in  the  development  process  was  a  goal  for  many  organizations  seeking 
some  update  about  the  software.  The  initial  intercept  showed  a  significant  difference 
between  the  original  and  new  model.  The  original  model  found  that  outsourcing 
decreased  visibility  whereas  the  new  model  showed  a  neutral  amount  of  visibility.  With 
the  new  model,  outsourcing  did  no  worse  at  this  visibility  than  those  organizations  not 
outsourcing.  Enterprise  (order  entry  and  website)  software  domains  and  outsourcing 
(design,  maintenance  and  fielding)  processes  were  indicating  variables  in  the  original 
model  but  not  in  the  new  model.  System  (avionic,  embedded,  and  device)  software 
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domains,  shrinkwrap  utility  software  domain,  component  (CASE  and  operating  system) 
software  domains,  outsourcing  (testing,  specification,  coding  and  configuration 
management)  processes,  total  product  outsourcing,  and  outsourcing  customizable 
common  products  were  indicating  variables  found  in  the  new  model  but  not  in  the 
original  model.  System  communication  software  domain,  development  tools  component 
software  domain,  enterprise  scripting  software  domain,  outsourcing  (reengineering  and 
software  engineer  support)  processes,  outsourcing  custom/speciabzed  products,  and  total 
process  outsourcing  were  common  indicating  variables  between  both  models.  System 
communication  software  domain,  outsourcing  custom/specialized  products,  and  total 
process  outsourcing  were  all  found  to  increase  visibility  in  both  models.  Development 
tools  component  software  domain  and  outsourcing  software  engineer  support  process 
showed  to  decrease  visibility  in  original  model,  but  in  the  new  model,  this  was  opposite 
with  these  variables  having  an  increasing  effect.  Enterprise  scripting  software  domain 
and  outsourcing  reengineering  process  showed  to  increase  visibility  in  the  original  model, 
but  in  the  new  model,  they  found  the  opposite  and  actually  decreased  visibility.  Given 
these  changes,  the  two  models  were  very  different  with  38  marks  separating  them.  The 
newer  model  did  have  higher  accuracy  with  a  R  value  of  0.87  and  proved  to  have  less 
noise  with  a  RMSE  value  of  0.66  making  it  the  model  of  choice. 

4.4.3.14  Control  of  Product  Consequence 

Controlling  the  product  consequence  referred  to  the  ability  for  the  organization  to 
induce  control  upon  a  software  project.  This  control  may  have  needed  to  inject  a  certain 
degree  of  accuracy  between  the  product  and  requirements  or  used  to  ensure  the  product 
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meets  a  certain  safety/testing  criteria.  The  intercept  from  the  old  model  showed  a  neutral 
control  outcome  which  would  be  at  least  equal  to  those  organizations  not  outsourcing. 
The  new  model  showed  a  complete  different  picture.  It  showed  that  controlling  the 
product  through  outsourcing  was  slightly  decreased.  Given  certain  outsourcing  factors, 
this  outcome  may  have  been  raised  significantly.  For  example,  those  choosing  total 
process  outsourcing  would  find  control  of  product  to  be  raised  to  a  slightly  increasing 
level.  System  device  software  domain,  enterprise  utility  software  domain,  domain 
framework  software  component  domain,  outsourcing  (reengineering,  application  support 
and  tool  support)  processes,  and  outsourcing  (COTS  and  customizable  common)  products 
proved  to  be  indicators  in  the  original  models  but  not  in  the  new  models.  Outsourcing 
(requirements,  testing  and  training)  processes  and  total  process  outsourcing  saw  their  way 
into  the  new  model  but  not  in  the  old  model.  Outsourcing  fielding  and  software  engineer 
support  processes  were  common  indicating  variables  found  in  both  models.  Outsourcing 
fielding  process  had  a  slight  decrease  effect  in  the  original  model  that  jumped 
dramatically  to  significantly  decreasing  this  consequence  in  the  new  model.  Outsourcing 
software  engineer  support  process  saw  in  the  original  model  to  decrease  this  control,  but 
in  the  new  model,  this  indicating  variable  was  found  correlated  with  an  increase  of  the 
control  of  the  product.  The  two  models  were  similar  with  16  marks  separating  them. 
The  newer  model  did  have  higher  accuracy  with  a  R"  value  of  0.48  and  proved  to  have 
less  noise  with  a  RMSE  value  of  0.99.  The  new  model  was  the  model  of  choice  due  to 
the  fact  it  did  a  better  job  explaining  the  outcome  and  was  built  with  the  latest  data. 
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4.4.3.15  Change  Cost  Consequence 

Change  cost  consequence  concerned  itself  with  goals  to  reduce  the  dollar  amount 
charged  for  changes  in  the  software  project.  These  changes  normally  resulted  in  change 
in  customer  requirements  or  hardware  restrictions.  Of  course,  the  changes  were  not 
restricted  to  just  these  types  alone.  Both  models  started  with  an  intercept  placing  this 
consequence  as  equal  to  those  organizations  not  outsourcing.  As  always,  this  intercept 
could  have  been  adjusted  depending  upon  which  indicating  variables  were  being  used. 
For  example,  using  the  new  model,  those  who  are  practicing  total  product  outsourcing 
found  a  significant  increase  in  this  cost,  but  those  who  were  practicing  total  process 
outsourcing  found  a  significant  decrease  for  this  consequence.  Those  indicating  variables 
found  in  the  original  model  but  not  in  the  new  model  included:  system  avionics  software 
domain,  shrinkwrap  internet  software  domain,  operating  system  component  software 
domain,  enterprise  (accounting,  manufacturing,  order  entry  and  website)  software 
domains,  outsourcing  software  engineer  support  process,  and  outsourcing  COTS 
products.  Those  indicating  variables  found  in  the  new  model  but  not  in  the  original 
included:  shrinkwrap  utility  software  domain,  component  class  library  software  domain, 
outsourcing  coding  process,  outsourcing  customizable  common  products,  and  total 
process  outsourcing.  No  common  indicating  variable  were  found  between  the  models. 
The  two  models  were  somewhat  similar  with  20  marks  separating  them.  Both  models  did 
have  similar  accuracy  figures  with  a  R“  approximate  value  of  0.36,  but  the  old  model 
proved  to  have  less  noise  with  a  RMSE  value  of  1.08.  The  new  model  was  the  model  of 
choice  due  to  the  fact  it  shared  the  same  accuracy  level  but  was  built  with  the  latest  data. 
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4.4.3.16  Language  Culture  Problem  Consequence 

Language/culture  problems  consequence  referred  to  the  risk  of  increasing 
communication  problems  due  to  a  difference  between  organizations  especially  those 
organizations  from  different  nations.  There  was  a  slight  difference  between  the  original 
and  new  model.  The  original  started  out  with  a  neutral  outcome  meaning  it  was  exactly 
the  same  as  those  who  decided  not  to  outsource.  The  new  model,  however,  showed  to 
have  a  slightly  increase  amount  of  these  types  of  problems  due  to  outsourcing.  System 
avionics  software  domain,  enterprise  (accounting  and  order  entry)  software  domains, 
outsourcing  (requirements  and  software  engineer  support)  processes,  and  outsourcing 
customizable  common  products  were  found  in  the  original  model  but  not  in  the  new 
model.  Embedded  system  software  domain,  component  (domain  framework  and  class 
library)  software  domains,  enterprise  payroll  software  domain,  outsourcing 
(reengineering,  application  support,  specifications  and  configuration  management) 
processes  and  total  process  outsourcing  were  all  indicator  variables  included  in  the  new 
model  but  not  the  old  model.  Shrinkwrap  utility  software  domain,  operating  system 
component  software  domain,  outsourcing  (design  and  fielding)  processes  and  outsourcing 
custom/specialized  products  were  common  in  both  models.  Shrinkwrap  utility  software 
domain  showed  an  increasing  effect  in  the  old  model  but  showed  a  decreasing  effect  in 
the  new  model.  Operating  system  component  software  domain  showed  a  small 
decreasing  effect  in  old  model  but  a  significant  decreasing  effect  in  the  new  model. 
Outsourcing  design  process  correlated  with  a  small  increasing  effect  in  original  model  but 
a  significant  decreasing  effect  in  the  new  model.  Outsourcing  fielding  process  saw  a 
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small  decrease  in  the  old  model  but  an  increase  in  the  new  model.  Outsourcing 
custom/specialized  products  stayed  consistent  with  a  small  decrease  through  out  both 
models.  The  two  models  were  somewhat  separated  with  25  marks  between  them.  The 
newer  model  did  have  higher  accuracy  with  a  R  value  of  0.61,  but  the  old  model  had  less 
noise  with  a  RMSE  value  of  0.62.  The  new  model  was  the  model  of  choice  due  to  the 
fact  it  did  a  much  better  job  explaining  the  outcome  and  was  built  with  the  latest  data. 

4.4.3.17  Turf  War  Consequence 

Turf  war  consequence  referred  to  the  friction  found  between  different  teams 
striving  to  complete  their  own  independent  task.  Teamwork  in  both  the  contractor  and 
customer  organizations  was  the  overall  goal.  This  consequence  described  the  effects 
outsourcing  had  on  this  teamwork  goal.  The  initial  intercept  showed  a  significant 
difference  between  the  original  and  new  model.  The  original  model  found  that 
outsourcing  kept  the  turf  war  neutral  which  was  equivalent  to  those  not  outsourcing 
whereas  the  new  model  showed  a  slightly  increased  amount  of  turf  war.  Indicating 
variable  selection  such  as  outsourcing  software  engineer  support  process  would  have 
decrease  affect  on  the  outcome.  System  avionic  software  domain,  component  (class 
library  and  operating  system)  software  domains,  enterprise  (accounting  and 
manufacturing)  software  domains,  outsourcing  (reengineering,  application  support, 
specification  and  tool  support)  processes,  and  outsourcing  COTS  products  were 
indicating  variables  in  the  original  model  but  not  in  the  new  model.  System  device 
software  domain,  enterprise  scripting  software  domain,  outsourcing  (design  and 
configuration  management)  processes,  outsourcing  customizable  common  products,  and 


119 


AFIT/GCS/ENG/04-16 


total  process  outsourcing  were  indicating  variables  found  in  the  new  model  but  not  in  the 
original  model.  System  embedded  software  domain,  shrinkwrap  internet  software 
domain,  and  outsourcing  software  engineer  support  process  were  common  indicating 
variables  between  both  models.  System  embedded  software  domain  had  a  consistent 
increasing  effect  on  turf  war  consequence.  Shrinkwrap  internet  software  domain  and 
outsourcing  software  engineer  support  process  showed  to  have  an  increasing  effect  in  the 
old  model  and  a  decreasing  effect  in  the  new  model.  The  models  were  different  with  24 
marks  separating  them.  The  older  model  did  have  higher  accuracy  with  a  R“  value  of 
0.57  and  proved  to  have  less  noise  with  a  RMSE  value  of  0.86  making  it  the  model  of 
choice  despite  being  built  with  older  data. 

4.4.3.18  Failure  Likelihood  Consequence 

Failure  likelihood  consequence  concerned  itself  with  outright  project  failure.  It 
was  assumed  by  many  organizations  that  outsourcing  may  be  a  prime  reason  for  project 
failure.  Figures  have  shown  quite  the  opposite,  and  in  some  cases,  actually  decreased  the 
chance  of  project  failure.  Both  models  started  with  an  intercept  placing  this  consequence 
as  equal  or  slightly  less  than  neutral.  This  would  imply  that  those  organizations  not 
outsourcing  had  about  the  same  result  with  project  failure.  Those  indicating  variables 
found  in  the  original  model  but  not  in  the  new  model  include:  system  device  software 
domain,  shrinkwrap  internet  software  domain,  and  outsourcing  configuration 
management  process.  Those  indicating  variables  found  in  the  new  model  but  not  in  the 
original  include:  shrinkwrap  (business  and  utility)  software  domains,  component  class 
library  software  domain,  enterprise  (accounting  and  scripting)  software  domains, 
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outsourcing  (testing,  maintenance  and  fielding)  processes,  and  outsourcing  customizable 
common  products.  Outsourcing  (requirements,  reengineering,  tool  support  and  software 
engineer  support)  processes  and  outsourcing  COTS  products  were  common  between  both 
models.  Outsourcing  (requirements  and  reengineering)  processes  showed  to  have  a 
decreasing  effect  in  the  old  model  but  an  increasing  effect  in  the  new  model. 
Outsourcing  (tool  support  and  software  engineer  support)  processes  and  COTS  products 
showed  to  have  an  increasing  effect  in  the  old  model  but  a  decreasing  effect  in  the  new 
model.  The  two  models  were  somewhat  different  with  28  marks  separating  them.  The 
newer  model  did  have  higher  accuracy  with  a  R  value  of  0.75  and  proved  to  have  less 
noise  with  a  RMSE  value  of  0.87.  The  new  model  was  the  model  of  choice. 

4.4.3.19  Response  to  Customer  Consequence 

Increasing  the  response  to  the  customer  was  a  popular  goal  for  many 
organizations  regardless  of  outsourcing  plans.  The  initial  intercept  showed  a  small 
difference  between  the  original  and  new  model.  The  original  model  found  that 
outsourcing  could  slightly  improve  this  consequence  whereas  the  new  model  showed 
only  a  neutral  outcome.  In  either  case,  outsourcing  did  no  worse  at  this  response  than 
those  organizations  not  outsourcing.  Enterprise  order  entry  software  domain  was  the 
only  indicating  variable  found  in  the  original  model  but  not  in  the  new  model. 
Shrinkwrap  (business  and  internet)  software  domains,  class  library  component  software 
domain,  enterprise  (accounting  and  scripting)  software  domains,  outsourcing 
(requirement,  design,  reengineering,  training  and  software  engineer  support)  processes, 
total  product  outsourcing,  and  total  process  outsourcing  were  indicating  variables  found 
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in  the  new  model  but  not  in  the  original  model.  Component  domain  framework  software 
domain,  outsourcing  (maintenance  and  application  support)  processes,  and  outsourcing 
(custom/specialized  and  customizable  common)  products  were  common  indicating 
variables  between  both  models.  Domain  framework  component  software  and 
outsourcing  (custom/specialized  and  customizable  common)  products  found  to  alternate 
from  decreasing  effect  in  the  old  model  to  increasing  effect  in  the  new  model.  Likewise 
outsourcing  maintenance  had  an  increasing  effect  in  the  old  model  to  a  decreasing  effect 
in  the  new  model.  Outsourcing  application  support  retained  a  decreasing  effect 
throughout  both  models.  The  two  models  were  somewhat  different  with  23  marks 
separating  them.  The  newer  model  did  have  higher  accuracy  with  a  R  value  of  0.80  and 
less  noise  with  a  RMSE  value  of  0.68  thus  making  it  the  choice  model  to  use. 

4.4.3.20  Response  to  Organization  Consequence 

Response  to  organization  consequence  concerned  itself  with  increasing  the 
response  to  organization’s  standards  and  expectations.  Many  organizations  wanted  their 
outsourced  organizations  to  respond  to  their  demands  rather  than  being  influenced  by 
other  sources.  Both  models  started  with  an  intercept  placing  this  consequence  as  equal  or 
slightly  less  than  neutral.  This  meant  that  those  seeking  response  from  a  total  in-house 
software  development  shop  had  similar  results.  Those  indicating  variables  found  in  the 
original  model  but  not  in  the  new  model  included:  enterprise  order  entry  software 
domain  and  outsourcing  (maintenance,  application  support  and  coding)  processes.  Those 
indicating  variables  found  in  the  new  model  but  not  in  the  original  included:  system 
communication  software  domain,  component  class  library  software  domain,  shrinkwrap 
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utility  software  domain,  enterprise  accounting  software  domain,  outsourcing 
(requirement,  training  and  fielding)  processes,  and  total  process  outsourcing. 
Outsourcing  software  engineer  support  process  was  the  only  common  indicating  variable 
found  between  both  models.  Outsourcing  software  engineer  process  showed  to  have  a 
decreasing  effect  in  the  old  model  but  a  significant  increasing  effect  in  the  new  model. 
The  two  models  were  somewhat  similar  with  17  marks  separating  them.  The  newer 
model  did  have  higher  accuracy  with  a  R  value  of  0.56  and  proved  to  have  less  noise 
with  a  RMSE  value  of  0.97.  The  new  model  was  the  model  of  choice. 

4.4.3.21  SODS2  Consequence  Models  Summary 

These  models  provided  us  an  overview  of  the  new,  used,  and  combined  data  sets. 
With  few  exceptions,  the  new  data  models  appeared  to  be  more  accurate  by  the  RMSE 
and  correlation  values  thus  did  a  better  job  explaining  the  variation  from  response  to 
response.  By  providing  the  user  the  ability  to  view  results  from  all  three  sets,  the  user 
have  a  much  thorough  result  set  along  with  the  accuracy  provided  by  the  NN’s.  This 
model  quality  evaluation  was  based  on  comparison  of  model  outputs  to  data  used  to 
create  the  models.  Later  in  this  chapter’s  validation  section,  validation  uses  unprocessed 
survey  data  to  establish  external  model  validity. 

4.4.4  Regression  Model  Summary 

This  section  covered  the  regression  model  implementation  in  regards  to  the 
SODS2  development.  The  following  three  goals  for  this  section  were  achieved: 

1 .  Modeling  technique  validation, 

2.  Analysis  on  differences  between  models,  and 
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3.  Explanation  of  SODS2  consequence  models. 

4.5  Neural  Network  Implementation 

The  regression  models  provided  an  understandable,  linear  meaning  to  software 
outsourcing  and  its  consequences.  But  these  relationships  between  the  input  variables 
found  in  the  above  models  were  not  always  linear.  As  discussed  in  Chapter  Two,  these 
models  along  with  the  NN  learning  mechanism  provided  a  more  complete  decision 
support  tool.  This  section  traverses  the  NN  implementation  for  SODS2  by  breaking  it  up 
in  the  following  subsections. 

4.5.1  Neural  Network  Design  Portion 

The  Neurosolutions  tool  as  discussed  in  Chapter  Three  was  important  for  the 
design,  training,  testing  and  validation  of  the  NN’s.  One  of  the  reasons  for  this  tool 
selection  was  its  quality  support  staff.  This  research’s  primary  Neurosolutions  support 
representative,  Mr.  Gary  Lynn,  with  over  1 1  years  working  with  Neurosolutions  holding 
a  Michigan  State  University  Computer  Science  Master’s  Degree,  advised  that  this 
research  use  MLP  with  backpropagation  following  a  function  approximation  architecture. 
This  meant  that  the  input  axon  for  the  hidden  layer,  a  function  that  compresses  all  of  the 
inputs  into  the  hidden  layer  input,  used  a  nonlinear  tangent  hyperbolic  function  to 
perform  such  mappings.  This  function  provided  the  NN  the  ability  to  make  soft  non¬ 
linear  decisions.  Neurosolutions  referred  to  this  as  the  TanhAxon.  The  output  axon  (the 
function  that  messages  the  final  output  signal  from  the  hidden  layer)  used  a  bias  (linear) 
function.  This  provided  the  NN  a  degree  of  adaptability  for  the  output  when  the  NN  was 
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forced  to  interpolate  because  of  being  trained  with  a  relative  low  amount  of  training  data. 
The  learning  algorithm  used  conjugate  gradient  method  because  from  Lynn’s  experience 
this  particular  learning  algorithm  proved  to  be  faster  at  training.  One  hidden  layer  was 
used  due  to  the  limited  amount  of  data  in  regards  to  the  high  number  of  input  variables. 
This  NN  design  provided  this  research  a  solid  fit  given  its  problem  according  to  the 
experts  at  Neurosolutions. 

4.5.2  Cross  Validation  Portion 

Data  was  configured  and  saved  in  a  tab  delimited  text  file  for  cross  validation. 
Using  this  text  file,  Neurosolutions  provided  an  easy  graphical  display  for  selecting  input 
and  output  columns  found  in  the  file.  Those  columns  selected  for  input  were  fed  into 
Neurosolutions  as  input  variables,  and  those  columns  selected  for  output  were  fed  as 
output  variables.  A  randomization  function  allowed  the  data  rows  to  be  fully 
randomized.  This  was  highly  recommended  by  Lynn  to  help  destroy  all  possible  order 
found  within  the  data  sets  so  that  such  order  did  not  interfere  with  training.  Selecting 
cross  validation  sets  as  mentioned  in  Chapter  Two  was  also  provided  through  graphical 
displays.  Neurosolutions  would  break  the  rows  down  into  percentages  to  allow  the 
selection  of  such  cross  validation  data.  During  actual  training,  the  tool  used  these  probe 
hooks  for  displaying  the  progress  of  the  training  versus  cross  validation.  The  NN 
performance  measurements  (specifically  the  Mean  Square  Error  (MSE)  and  Correlation 
(R)  Value)  provided  accuracy  measure  for  the  NN  against  both  the  training  and  cross 
validation  data  set.  The  data  graph  that  compared  the  training  MSE  line  with  the  cross 
validation  MSE  line  proved  most  useful.  This  data  graph  copied  exactly  the  intent  behind 
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Figure  18  MLP  NN  Training  Error  Plot  [39].  Appendix  E  contains  the  screen  shots  from 
Neurosolutions  taken  after  each  consequence  cross  validation.  The  performance 
measures  and  error  plots  (along  with  the  basic  graphical  design  of  the  NN)  were  captured 
within  each  screen  shot. 

4.5.3  Neural  Network  Data  Problem 

Upon  review  of  the  cross  validation  data  and  associated  performance 
measurements,  a  problem  was  quickly  noticed.  Due  to  the  limited  number  of  available 
data  and  a  pre-existing  interpolation  problem  as  discussed  in  the  regression  models,  the 
cross  validation  data  needed  to  somehow  be  included  within  the  training  data. 
Comparing  all  cross  validation  screen  shots,  it  was  reasonable  to  believe  that  overtraining 
could  be  kept  low  by  stopping  training  at  50  epochs.  By  averaging  out  the  cross 
validation/over- training  minimum  epoch  point  for  each  error  plot,  one  could  assure  that 
overtraining  would  be  minimized  if  training  was  stopped  at  the  research  average  of  50 
epochs.  Lynn  verified  that  this  was  a  sound  methodology  for  solving  my  interpolation 
problem  caused  by  having  a  small  amount  of  training  data. 

4.5.4  Neural  Network  Training  Without  Cross  Validation  Portion 

Applying  such  methodology  meant  that  this  research  retrained  each  NN  with  all 
rows  of  data  added.  During  this  retraining,  the  training  data  set  perfonnance 
measurement  probe  hook  provided  the  NN  accuracy  infonnation  and  using  a  new  data 
graph  probe  hook  for  comparing  the  actual  survey  outputs  vs.  NN  estimated  outputs 
demonstrated  a  nice  overview  of  the  NN  training.  These  values  and  graphs  were  placed 
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in  the  screen  shots  included  in  Appendix  F.  One  Neurosolutions’  learning  hurdle  dealt 
with  the  way  the  tool  saved  its  weight  files.  Unless  configured  correctly,  it  would  save 
dynamically  the  best  weights  associated  with  a  history  of  training  runs.  This  caused  a 
problem  when  the  goal  was  to  capture  the  training  screen  shot  with  the  weights  currently 
used  within  the  NN.  By  explicitly  saving  the  initial  weight  files  along  with  the  final 
weight  files,  the  training  episode  associated  with  a  given  screen  shot  could  be 
reproduced.  This  infonnation  was  from  the  excellent  help  support  staff  there  at 
Neurosolutions  and  a  crucial  discovery  that  made  building  these  NN  that  much  easier. 

4.5.5  Neural  Network  Interface  Portion 

With  the  above  methodology  for  building  NN  in  place,  20  NN  were  reasonably 
designed  and  trained.  The  only  remaining  portion  dealt  with  interfacing  the  20  NN  with 
SODS2  during  run  time.  As  previously  documented,  one  of  Neurosolutions’  most 
beneficial  features,  was  the  ability  for  it  to  create  20  DLL’s  and  associated  weight  files 
using  the  MS  Visual  C++  compiler  version  5.0  to  7.0.  The  author  used  the  MS  Access 
VB  sample  shell  provided  by  Neurosolutions  to  understand  how  SODS2  would  connect 
to  these  DLL  files. 

4.5.6  NN  Implementation  Summary 

Twenty  consequence  NN’s  were  designed,  trained  and  created  by  the 
Neurosolutions  under  the  expert  guidance  of  Lynn.  The  interfacing  portion  used  this  tool 
to  create  20  DLL  files.  These  DLL  files  would  allow  MS  Access  VB  to  interface  directly 
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with  the  20  NN’s.  At  this  point  with  the  NN’s  and  models  built,  attention  was  turned  to 
the  development  of  SODS2. 

4.6  SODS2  Implementation 

This  implementation  section  covered  building  the  actual  SODS2  application.  This 
phase  of  the  implementation  will  be  broken  into  the  following  seven  subphases. 

4.6.1  SODS2  Development  Tool 

During  the  NN  interfacing  portion,  this  researcher  was  happy  to  discover  that  the 
Neurosolutions  DLL  files  could  actually  be  called  from  inside  MS  Access.  This  caused 
yet  another  change  in  methodology  because  the  plan  was  to  use  Togethersoft  Java  in 
SODS2  development.  By  using  MS  Access  VB  as  the  developer’s  platform,  MS  Access 
would  not  only  be  used  to  store  and  organize  the  data,  but  would  house  the  application 
along  with  this  data.  This  promised  an  easier  fielding  of  the  application  and  less 
configuration  management  headaches.  The  disadvantage  of  this  decision  was  that  MS 
Access  VB  lacked  the  maintainability  and  software  evolution  that  would  have  been 
offered  through  a  fully  dependent  programming  language  such  as  Java,  but  this  research 
found  that  software  outsourcing  was  rapidly  changing  so  the  lifespan  of  such  application 
would  be  a  few  years  at  most. 

Using  MS  Access  VB,  the  application  was  built  meeting  all  the  high  level 
requirements  found  in  Table  26.  A  more  detailed  outline  of  the  SODS2  windows  were 
included  in  Appendix  I  which  showed  how  each  requirement  was  incorporated  in  the 
SODS2  tool. 
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4.6.2  SODS2  Validation  Phase 

The  SODS2  validation  consisted  of  120  scenarios  created  from  six  new  survey 
responses  gathered  since  the  release  of  this  research  analysis  data.  Due  to  the  fact  these 
responses  were  not  used  in  the  formulation  of  either  the  regression  models  or  NN’s, 
unbiased  results  were  captured.  SODS2  validation  used  its  scenario  results  from  the  New 
Data  Regression  (NDR),  Old  Data  Regression  (ODR),  and  Combined  Data  Regression 
(CDR)  models  along  with  results  from  the  NN  learning  mechanism.  These  results  were 
then  compared  against  the  actual  survey  responses.  Figure  41  illustrated  such 
comparisons  as  it  relates  to  the  cost  consequence. 
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This  chart  was  created  from  the  following  data  found  in  Table  28. 


Table  28  Validation  Sample  Table 


Survey 

# 

Actual 

Response 

Neural 

Network 

NDR 

ODR 

CDR 

Diff. 

NN 

Diff. 

NDR 

Diff. 

ODR 

Diff. 

CDR 

1 

3.00 

4.19 

2.32 

6.34 

4.07 

1.19 

0.68 

3.34 

1.07 

2 

3.00 

1.12 

5.06 

7.36 

4.28 

1.88 

2.06 

4.36 

1.28 

3 

2.00 

3.52 

2.32 

5.61 

2.77 

1.52 

0.32 

3.61 

0.77 

4 

3.00 

4.47 

5.53 

4.95 

4.16 

1.47 

2.53 

1.95 

1.16 

5 

2.00 

4.91 

2.32 

5.42 

4.04 

2.91 

0.32 

3.42 

2.04 

6 

2.00 

0.17 

3.70 

3.57 

1.89 

1.83 

1.70 

1.57 

0.11 

Total  Difference  (Neural  Network): 

10.80 

Total  Difference  (New  Data  Regression): 

7.61 

Total  Difference  (Old  Data  Regression): 

18.25 

Total  Difference  (Combined  Data  Regression): 

6.43 

This  sample  illustrated  each  respondent’s  (one  through  six)  actual  responses  compared  to 
the  estimated  outputs  calculated  with  each  learning  mechanism.  The  distance  from  the 
actual  response  and  each  output  was  calculated  and  inserted  in  each  of  the  Differences 
(Diff.)  columns.  Then  these  distances  were  summed  together  at  the  bottom  of  the 
sample’s  tables  resulting  in  the  total  difference.  This  was  graphically  illustrated  in  the 
sample’s  chart.  In  respondent’s  one  cost  scenario,  the  NDR  performed  the  best  in 
actually  predicting  the  outcome  while  in  respondent’s  six  cost  scenario,  the  NDR 
performed  the  worst.  All  120  scenarios  were  included  in  Appendix  H. 

After  the  120  scenarios,  a  summarization  was  calculated  from  the  total  differences 
and  inserted  at  the  end  of  Appendix  H.  CDR  slightly  edge  NN  for  best  perfonnance  on 
these  six  data  points.  NDR  and  ODR  were  similarly  close  with  significantly  worse 
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performance.  Interpolating  effects  due  to  the  small  amount  of  data  used  in  building  the 
NDR  and  ODR  probably  accounted  for  this  performance.  Another  indicator  of 
interpolation  was  the  extreme  estimates  of  greater  than  seven  or  less  than  one.  The 
results  of  this  summary  were  listed  in  Figure  42. 
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Figure  42  Validation  Summary 

Each  learning  mechanism  performed  reasonably  well  given  the  limited  amount  of 
data  this  research  used  to  build  all  four  mechanisms.  This  overall  performance,  as 
illustrated  above,  was  well  within  the  expectations  of  this  research.  The  NN  and  CDR 
used  all  the  training  data  captured  in  the  combined  survey  data  set;  thus,  proved  to 
outperform  the  NDR  and  ODR  even  though  these  models  showed  better  overall 
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performance  within  their  own  data  sets.  The  driving  push  behind  such  results  was  the 
importance  of  capturing  as  much  data  as  possible  over  a  wide  range  of  possible  scenarios. 
This  should  help  alleviate  interpolation  effects.  Though  NDR  and  ODR  fell  shortly 
behind,  all  resulting  learning  mechanisms’  performances  were  reasonably  accurate  given 
such  fluctuation  in  the  validation  survey  set.  SODS2  met  this  validation  criteria  in  two 
different  ways:  1)  SODS2  learning  mechanisms’  perfonnance  measures  provided  decent 
estimated  outputs,  and  2)  SODS2  perfonned  brilliantly  in  processing  the  output  for  the 
120  validating  scenarios  that  used  several,  varying  input  combinations. 

4.6.3  SODS2  Implementation  Summary 

This  section  covered  the  SODS2  implementation  steps.  Each  of  the  seven  sub¬ 
phases  linked  to  a  subset  of  requirements  laid  out  in  the  methodology.  The  tool  was  built 
with  MS  Access  VB  with  the  intent  of  providing  a  user  friendly,  window  type 
application.  The  point  and  click  input  features  and  associated  help  tags  gave  the  user  an 
efficient  interface  to  enter  their  respective  software  outsourcing  input  parameters  and 
navigate  to  viewing  the  associated  reports.  The  reports  were  broken  into  an  easy  to 
decipher  scale  providing  the  user  the  predicted  outcomes  based  upon  their  input 
selections. 

This  application  was  validated  with  the  newest  survey  data  yet  to  be  inserted  into 
either  of  the  SODS2  learning  mechanisms.  This  survey  data  allowed  the  creation  of  120 
scenarios  based  upon  what  the  respondent  might  have  entered  given  access  to  SODS2. 
By  comparing  these  actual  outputs  with  the  SODS2  learning  mechanisms’  estimated 
outputs,  one  could  see  how  this  decision  support  tool  would  have  performed  given  such 
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survey  respondents  inputs  compared  to  real  world  output.  The  SODS2  learning 
mechanisms  were  highly  adaptable  at  estimating  such  outputs.  At  this  stage,  SODS2  was 
completed  and  all  associated  requirements  were  fully  met. 

4.7  Summary 

This  chapter  described  the  implementation  results  for  this  research.  This  effort 
began  with  the  transport  of  new  data  into  the  MS  Access  database  housing  the  previous 
collected  survey  data.  The  data  was  then  analyzed  using  statistical  methods  such  as  T- 
testing  and  regression  model  building.  Numerous  differences  with  only  a  small  amount 
of  similarities  were  found  between  the  old  and  new  surveys.  The  analysis  continued  with 
the  creation  of  the  regression  models.  After  the  modeling  techniques  were  validated  and 
used  to  create  the  goal  realization  models,  differences  were  again  noticed  from  the  old 
survey  data  goal  realization  models  created  in  Hermann’s  dissertation  and  the  new  goal 
realization  models  created  in  this  research.  With  these  differences  and  a  change  in  the 
stepwise  entry/removal  statistics,  this  research  proceeded  in  creating  60  consequence 
regression  models  from  the  new,  used  and  combined  data  sets.  From  the  creation  of  these 
models,  the  attention  turned  toward  the  NN  implementation  phase.  Neurosolutions  tool 
along  with  its  expert  support  staff  provided  this  research  properly  designed  and  trained 
NN  for  each  consequence.  The  tool  also  paved  the  way  for  interfacing  with  SODS2  by 
means  of  DLL  files.  With  the  NN’s  and  models  built,  the  SODS2  implementation  was 
initiated.  SODS2  was  successfully  built  with  all  requirements  fully  achieved  and 
validated  with  120  scenarios. 
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5.  Conclusion  and  Future  Work 

5.1  Introduction 

This  chapter  will  summarize  this  research  along  with  providing  conclusions  and 
suggestions  for  future  work.  The  results  will  sum  up  the  contributions  brought  forth  by 
this  work.  After  these  contributions  are  summarized,  this  chapter  concludes  with  a  vision 
for  future  work. 

5.2  Conclusions  and  Contributions 

Many  new  discoveries  were  noted  along  the  way  in  developing  Software 
Outsourcing  Decision  Support  tool  version  2  (SODS2).  The  first  discovery  came  through 
the  analysis  of  the  new  survey  data.  It  was  previously  assumed  that  this  data  would  share 
many  common  similarities  with  a  sprinkling  of  differences  between  the  analysis  and 
consequence  rules  used  in  Hermann’s  first  version  of  the  tool  (SODS1).  This  was  hardly 
the  case.  These  differences  slightly  repositioned  this  research’s  methodology.  Instead  of 
building  the  20  stepwise  regression  consequence  models  from  only  the  combined  survey 
data,  now,  they  should  be  built  using  all  three  sets  of  survey  data,  the  new,  used  and 
combined  totaling  60  regression  models.  Discovering  the  fact  that  outsourcing 
experience  is  rapidly  changing  in  a  significant  positive  way  was  this  research’s  first 
contribution. 

The  next  major  discovery  dealt  with  creating  a  new  methodology  for  extending  the 
current  SODS  1  decision  support  tool  to  allow  the  addition  of  neural  network  technology. 
Because  of  the  accuracy  that  neural  network  technology  promised  to  the  SODS2  user,  a 
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change  in  the  stepwise  entrance  /  removal  statistics  from  95%  /  90%  confidence  level  to 
75%  /  75%  confidence  level  was  allowed.  Expanding  this  stepwise  entrance  /  removal 
statistics  caused  dramatic  inclusions  of  new  variables  into  all  60  regression  models  used 
by  SODS2.  This  change  improved  the  understandability  about  the  consequence  outputs 
as  they  relate  to  even  a  greater  number  of  variables  selected  for  insertion  into  the  models. 
However,  with  this  expansion  of  the  stepwise  entrance  /  removal  statistics,  more  noise 
was  allowed  into  the  new  models  so  neural  networks  had  to  be  used  to  provide  SODS2 
users  the  needed  accuracy.  This  meant  that  this  research  designed  and  employed  a  new 
methodology  for  adding  neural  networks  along  with  the  regression  models  to  the  SODS2 
tool.  As  a  result  of  such  methodology,  20  Multi-Layer  Perceptron  (MLP)  with 
backpropagation  neural  networks  were  designed  and  verified  by  a  notable  neural  network 
expert.  New  methods  for  applying  cross  validation  to  minimize  the  amount  neural 
network  overtraining  and  overfitting  were  also  designed  and  applied.  These  new 
methods  allowed  the  neural  network  to  be  trained  with  all  of  the  collected  input  data,  but 
by  stopping  the  training  based  upon  an  average  epoch  calculated  during  cross  validation, 
overtraining  and  overfitting  were  minimized.  Such  neural  network  training  methods  were 
reviewed  and  verified  by  Lynn.  As  a  result,  these  20  neural  networks  provided  the 
accuracy  as  promised  with  a  decent  ability  of  explaining  the  varying  combined  survey 
outputs.  This  SODS2  addition,  of  using  neural  network  artificial  intelligence  along  with 
the  three  separate  sets  of  regression  models,  provided  outsourcing  decision  makers  an  in- 
depth  consequence  output  regarding  their  proposed  outsourced  project.  This  new  SODS2 


135 


AFIT/GCS/ENG/04-16 


methodology  for  adding  neural  networks  with  their  associated  changes  was  this 
research’s  second  major  contribution. 

With  the  new  methodology  designed,  this  research  proceeded  to  build  and  validate 
SODS2.  During  the  development  of  the  tool,  much  of  the  focus  was  on  making  it  user 
friendly.  Building  the  application  inside  the  MS  Access  survey  database  provided  the 
user  an  easy  all-in-one  portable  tool.  SODS2  development  language,  MS  Access  Visual 
Basic,  provided  the  means  for  adding  a  window  graphical  interface  with  point  /  click  type 
inputs.  User  tool  guidance  was  included  through  help  labels  and  smart  description  tags. 
The  requirements  shown  in  Chapter  Three’s  Table  26  were  fully  met  by  SODS2. 

After  successful  completion  of  SODS2,  recent  survey  data  was  collected  from  the 
internet  which  was  used  to  generate  120  scenarios  for  SODS2  validation.  Such  survey 
data  was  not  used  in  the  creation  of  the  regression  models  nor  the  neural  networks; 
therefore,  providing  an  unbiased  validation  result  set  for  this  research.  Each  SODS2 
learning  mechanism,  Neural  Networks  (NN),  New  Data  Regression  (NDR),  Old  Data 
Regression  (ODR),  and  Combined  Data  Regression  (CDR),  aligned  their  estimates  fairly 
favorable  to  each  of  the  scenario’s  actual  output  as  noted  in  Appendix  H.  NN  and  CDR 
were  evenly  matched  with  the  best  overall  perfonnance  while  NDR  and  ODR  had  similar 
performance  and  were  not  far  behind.  Such  performances  resulted  from  interpolation 
problems  due  to  the  amount  of  data  used  to  create  NDR  and  ODR  versus  the  amount  of 
data  used  to  create  NN  and  CDR.  Such  validation  results  showed  the  value  that  SODS2 
could  provide  to  a  outsourcing  software  decision  maker.  With  the  SODS2  built  and 
validated,  a  third  and  important  contribution  was  added  to  this  field  of  study. 
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5.3  Future  Work 

During  the  development  of  SODS2,  over  128  regression  consequence  and  goal 
realization  stepwise  regression  models  were  created  and  analyzed.  The  new  survey  data 
and  models  were  analyzed  and  compared  against  the  findings  presented  in  Hennann’s 
dissertation.  As  previously  discussed,  the  real  world  software  outsourcing  knowledge 
was  found  to  be  changing.  Ongoing  analysis  of  survey  data  will  be  paramount  for  the 
evolution  of  software  outsourcing  knowledge.  In  attempts  to  provide  such  analysis, 
enhancements  to  the  online  survey  will  be  the  first  crucial  step  in  capturing  this 
knowledge.  The  survey  response  Likert  scale  can  be  adapted  so  that  a  graphical 
continuous  scale  can  be  used  in  its  replacement;  much  like  the  scale  used  to  adjust  a 
computer’s  volume.  This  would  allow  the  survey  respondents  to  slide  the  graphical  slider 
to  real  data  type  values  between  the  whole  numbers  of  one  through  five  or  one  through 
seven  respectively.  Along  with  this  improvement,  additional  changes  to  the  survey  could 
provide  respondents  an  opportunity  to  add  survey  responses  concerning  their  most  vivid, 
worst,  and/or  successful  software  outsourcing  project  experiences.  The  goal  with  this 
change  is  the  attempt  to  capture  as  much  infonnation  from  the  user  as  possible.  Another 
important  survey  change  deals  with  changing  software  domain  flags  as  categorical  type 
input  versus  specific  type  input.  Reducing  the  number  of  input  variables  should  increase 
the  accuracy  of  both  the  regression  models  and  neural  networks.  In  light  of  this, 
unfortunately,  additional  flags  may  be  required  to  capture  future  software  outsourcing 
trends  such  as  Application  Service  Providers  and/or  outsourcing  partnerships.  Given 
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such  changes,  the  drive  for  this  future  work  will  be  to  collect  as  much  software 
outsourcing  data  using  an  revised  online  survey  tool. 

Due  to  the  increase  of  such  new  survey  data,  future  experimentation  may  be 
created  to  isolate  the  first  cause  and  effect  software  outsourcing  relationship.  This 
research  was  purely  observational  with  only  opinions  offered  for  the  observed  software 
outsourcing  occurrences.  Experimentation  might  concentrate  on  a  very  small  subset  of 
an  outsourcing  type  and  research  to  see  if  a  cause  and  effect  relationship  can  be  found. 
While  the  over-arching  observational  tendencies  have  been  documented,  very  little  or 
possibly  no  work  has  ever  attempted  to  study  the  cause  and  effect  of  a  particular  subset  of 
software  outsourcing. 

Future  efforts  must  also  be  spent  on  enhancing  SODS2  to  be  more  dynamic. 
Neurosolutions  using  neural  network  technology  can  be  used  to  allow  a  SODS2  user  the 
ability  to  accept  a  constant  flow  of  incoming  new  neural  network  and  stepwise  regression 
update  packages.  This  enhancement  would  function  much  like  the  antivirus  software 
found  on  many  computers.  Once  a  considerable  amount  of  surveys  are  collected,  an 
auto-updating  SODS  server  will  need  to  be  created.  Its  purpose  includes: 

1 .  Automate  the  regression  model  and  neural  network  creation  with  reports  providing  a 
summary  of  the  changes  between  the  regression  models  and  neural  networks, 

2.  Allow  the  user  version  of  SODS  to  automatically  pull  new  model  and  neural  network 
packages  from  this  new  server  by  means  of  the  internet,  and 

3.  Provide  online  communications  for  SODS  users  informing  them  about  new  updates 
and  associated  analysis  reports,  giving  them  a  medium  to  voice  their  opinion. 

The  goals  for  this  tool  would  not  really  change,  but  the  process  of  capturing,  codifying 

and  evolving  software  outsourcing  knowledge  would  be  greatly  enhanced. 
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Appendix  A  -  Validation  Goal  Realization  Models 


The  following  two  models  validate  this  research’s  JMP  modeling  techniques  against  the  techniques  exercised 
during  Hermann’s  dissertation  to  ensure  both  techniques  are  the  same: 


Stepwise  Fit  -  Old  Survey  Data  -  Goal  Realizations  (Response  to  Organization) 

Response: 

Column  53 


Stepwise  Regression  Control 


Prob  to  Enter  0.050 

Prob  to  Leave  0.100 

Direction: 

Rules: 


Current  Estimates 

SSE  DFE 

56.525847  65 


MSE  RSquare 

0.8696284  0.2382 


Lock 

X 


X  Intercept 

whattypesystemsavionics{0-1 } 
whattypesystemsembedded{0-1 } 
whattypesystemscommunications{0-1} 
X  whattypesystemsdevice{0-1} 

whattypeshrinkbusiness{0-1} 
whattypeshrinkutilities{0-1 } 
whattypeshrinkinternet{0-1 } 
whattypecomponentdomain{0-1} 
whattypecomponentCASE{0-1 } 
whattypecomponentclass{0- 1 } 
whattypecomponentOS{0-1} 
whattypecomponentdevelopment{0-1} 
whattypeenterpriseacctng{0-1 } 
whattypeenterprisemanufact{0-1} 
whattypeenterprisepayroll{0-1 } 
whattypeenterpriseOES{0- 1 } 
whattypeenterprisescripting{0-1} 
whattypeenterpriseweb{0-1 } 
whatprocessrequirements{0-1} 
whatprocessdesign{0- 1 } 
whatprocesstesting{0- 1 } 
whatprocessmaintenance{0-1 } 
whatprocessreengineering{0-1} 
whatprocessappsuppt{0-1 } 
whatprocesstraining{0-1} 
whatprocessspecification{0-1 } 
whatprocessdocumentation{0-1 } 

X  whatprocesscoding{0-1} 

whatprocessfielding{0-1} 
whatprocessCM{0-1} 
whatprocesstoolsuppt{0-1} 

X  whatprocessSWEngSuppt{0-1} 

whatproductscustom{0-1 } 
whatproductsCOTS{0-1} 
whatproductscommoncust{0-1} 
whatproductsnone{0-1} 


Step  History 

Step  Parameter  Action 

1  whattypesystemsdevice{0-1}  Entered 

2  whatprocessSWEngSuppt{0-1}  Entered 

3  whatprocesscoding{0-1}  Entered 


RSquare  Adj 
0.2031 


Cp 

-4.692347 


AIC 

-5.75919 


Estimate  nDF 

1.81216578  1 

0  1 

0  1 

0  1 

0.58890374  1 

0  1 

0  1 

0  1 

0  1 

0  1 

0  1 

0  1 

0  1 

0  1 

0  1 

0  1 

0  1 

0  1 

0  1 

0  1 

0  1 

0  1 

0  1 

0  1 

0  1 

0  1 

0  1 

0  1 

-0.3961676  1 

0  1 

0  1 

0  1 

0.30481283  1 

0  1 

0  1 

0  1 

0  1 


SS 

0 

0.094899 

0.187166 

0.097644 

6.222779 

0.378797 

0.085522 

2.145915 

0.919992 

1.104542 

0.021969 

0.067513 

0.425571 

0.099378 

0.254533 

0.021969 

3.009277 

0.49241 

0.073168 

0.372773 

0.326579 

0.659481 

0.905359 

2.336888 

0.690519 

1.58186 

0.010011 

0.027099 

4.711611 

0.848116 

0.341952 

0.001773 

5.345948 

0.112019 

0.474052 

1.239521 

0.108289 


"Sig  Prob" 
0.0029 
0.0496 
0.0231 


Seq  SS 
9.252899 
3.712542 
4.711611 


RSquare 

0.1247 

0.1747 

0.2382 


"F  Ratio"  "Prob>F" 

0.000  1.0000 

0.108  0.7439 

0.213  0.6463 

0.111  0.7404 

7.156  0.0094 

0.432  0.5135 

0.097  0.7565 

2.526  0.1169 

1.059  0.3073 

1.276  0.2630 

0.025  0.8752 

0.077  0.7829 

0.485  0.4885 

0.113  0.7382 

0.289  0.5924 

0.025  0.8752 

3.599  0.0623 

0.562  0.4560 

0.083  0.7743 

0.425  0.5169 

0.372  0.5441 

0.755  0.3880 

1.042  0.3113 

2.760  0.1015 

0.791  0.3770 

1.843  0.1794 

0.011  0.9155 

0.031  0.8615 

5.418  0.0231 

0.975  0.3272 

0.390  0.5348 

0.002  0.9644 

6.147  0.0158 

0.127  0.7226 

0.541  0.4646 

1.435  0.2354 

0.123  0.7271 


Cp  p 

-0.301  2 

-1.999  3 

-4.692  4 
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Response  Column  53 
Actual  by  Predicted  Plot 
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1 


.5  1.0  1.5  2.0  2.5  3.0  3.5  4.0  4.5  5.0 
Column  53  Predicted  P=0.0005  RSq=0.24 
RMSE=0.9325 


Summary  of  Fit 


RSquare 
RSquare  Adj 
Root  Mean  Square  Error 
Mean  of  Response 
Observations  (or  Sum  Wgts) 

Analysis  of  Variance 


Source  DF 

Model  3 

Error  65 

C.  Total  68 


0.238226 

0.203067 

0.932539 

2.710145 

69 


Sum  of  Squares 
17.677052 
56.525847 
74.202899 


Mean  Square  F  Ratio 

5.89235  6.7757 

0.86963  Prob  >  F 

0.0005 


Lack  Of  Fit 


Source  DF 

Lack  Of  Fit  2 

Pure  Error  63 

Total  Error  65 


Sum  of  Squares 
0.470291 
56.055556 
56.525847 


Mean  Square  F  Ratio 

0.235146  0.2643 

0.889771  Prob  >  F 

0.7686 
Max  RSq 
0.2446 


Parameter  Estimates 


Term 

Estimate 

Std  Error 

t  Ratio  Prob>|t| 

Intercept 

2.3097148 

0.315128 

7.33  <.0001 

whattypesystemsdevice[1  -0] 

-1.177807 

0.4403 

-2.68  0.0094 

whatprocesscoding[1  -0] 

0.7923351 

0.340401 

2.33  0.0231 

whatprocessSWEngSuppt[1  -0] 

-0.609626 

0.245877 

-2.48  0.0158 

Effect  Tests 

Source 

Nparm  DF 

Sum  of  Squares 

F  Ratio 

Prob  >  F 

Whattypesystemsdevice 

1  1 

6.2227790 

7.1557 

0.0094 

Whatprocesscoding 

1  1 

4.7116115 

5.4180 

0.0231 

whatprocessSWEngSuppt 

1  1 

5.3459482 

6.1474 

0.0158 

RlzResponseOrg  =  2.31  +  (-1. 

.  1 8)sys-device  +  (-.61)proc-SWEngSup  +  (.79)proc-coding 

Exact  formula  included  in  dissertation  [11]. 
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For  additional  confidence,  revalidate  it  a  second  time  with  a  different  goal  realization: 


Stepwise  Fit  -  Old  Survey  Data  -  Goal  Realizations  (Response  to  Customer) 

Response: 

Column  53 

Stepwise  Regression  Control 

Prob  to  Enter  0.050 

Prob  to  Leave  0.100 

Direction: 

Rules: 


Current  Estimates 

SSE  DFE 

50.113114  61 


MSE  RSquare 

0.8215265  0.3134 


RSquare  Adj 
0.2684 


Cp 

5.0181432 


AIC 

-8.17455 


Lock 

X 


X  Intercept 

X  whattypesystemsavionics{0-1} 

whattypesystemsembedded{0-1 } 
whattypesystemscommunications{0-1} 

X  whattypesystemsdevice{0-1} 

whattypeshrinkbusiness{0-1} 
whattypeshrinkutilities{0-1 } 
whattypeshrinkinternet{0-1 } 
whattypecomponentdomain{0-1} 
whattypecomponentCASE{0-1} 
whattypecomponentclass{0- 1 } 
whattypecomponentOS{0-1} 
whattypecomponentdevelopment{0-1} 
whattypeenterpriseacctng{0-1 } 
whattypeenterprisemanufact{0-1} 
whattypeenterprisepayroll{0-1 } 

X  whattypeenterpriseOES{0-1} 

whattypeenterprisescripting{0-1} 
whattypeenterpriseweb{0-1 } 
whatprocessreq  u  irements{0- 1 } 
whatprocessdesign{0- 1 } 
whatprocesstesting{0- 1 } 
whatprocessmaintenance{0-1 } 
whatprocessreengineering{0-1} 
whatprocessappsuppt{0-1 } 
whatprocesstraining{0-1} 
whatprocessspecification{0-1 } 
whatprocessdocumentation{0-1 } 
whatprocesscoding{0- 1 } 
whatprocessfielding{0-1 } 
whatprocessCM{0-1} 
whatprocesstoolsuppt{0-1} 

X  whatprocessSWEngSuppt{0-1} 
whatproductscustom{0-1 } 
whatproductsCOTS{0-1} 
whatproductscommoncust{0-1} 
whatproductsnone{0-1} 


Step  History 

Step  Parameter  Action 

1  whatprocessSWEngSuppt{0-1}  Entered 

2  whattypesystemsdevice{0-1}  Entered 

3  whattypeenterpriseOES{0-1}  Entered 

4  whattypesystemsavionics{0-1}  Entered 


Estimate  nDF 

3.22599671  1 

-0.483542  1 

0  1 

0  1 

0.50514003  1 

0  1 

0  1 

0  1 

0  1 

0  1 

0  1 

0  1 

0  1 

0  1 

0  1 

0  1 

-0.7524053  1 

0  1 

0  1 

0  1 

0  1 

0  1 

0  1 

0  1 

0  1 

0  1 

0  1 

0  1 

0  1 

0  1 

0  1 

0  1 

0.42454695  1 

0  1 

0  1 

0  1 

0  1 


SS 

0 

3.299417 
0.732816 
0.039665 
4.534135 
0.017865 
0.000048 
1.489246 
1.277158 
0.297152 
1.258009 
0.080823 
0.414954 
0.071278 
0.005279 
1 .258009 
4.343437 
0.21359 
0.059232 
0.437746 
0.034148 
0.71897 
0.197592 
0.70859 
0.20126 
0.013362 
0.165089 
1.326795 
1.586029 
2.140979 
0.061198 
1.011529 
9.945977 
0.026689 
0.32977 
0.830003 
0.866997 


"F  Ratio"  "Prob>F" 

0.000  1.0000 

4.016  0.0495 

0.890  0.3491 

0.048  0.8282 

5.519  0.0221 

0.021  0.8842 

0.000  0.9940 

1.838  0.1803 

1.569  0.2152 

0.358  0.5519 

1.545  0.2187 

0.097  0.7566 

0.501  0.4818 

0.085  0.7710 

0.006  0.9369 

1.545  0.2187 

5.287  0.0249 

0.257  0.6142 

0.071  0.7908 

0.529  0.4700 

0.041  0.8404 

0.873  0.3538 

0.238  0.6278 

0.861  0.3573 

0.242  0.6246 

0.016  0.8998 

0.198  0.6577 

1.632  0.2064 

1.961  0.1666 

2.678  0.1070 

0.073  0.7874 

1.236  0.2707 

12.107  0.0009 

0.032  0.8587 

0.397  0.5308 

1.010  0.3188 

1.056  0.3082 


'Sig  Prob"  Seq  SS 

0.0028  9.585455 

0.0119  6.107857 

0.0378  3.879006 

0.0495  3.299417 


RSquare  Cp 

0.1313  15.196 

0.2150  9.7586 

0.2682  7.0355 

0.3134  5.0181 
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Response  Column  53 
Actual  by  Predicted  Plot 
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Column  53  Predicted  P=0.0001  RSq=0.31 
RMSE=0.9064 


Summary  of  Fit 


RSquare  0.313376 

RSquare  Adj  0.268352 

Root  Mean  Square  Error  0.906381 

Mean  of  Response  2.651515 

Observations  (or  Sum  Wgts)  66 


Analysis  of  Variance 


Source  DF 

Model  4 

Error  61 

C.  Total  65 


Sum  of  Squares 
22.871735 
50.113114 
72.984848 


Mean  Square 
5.71793 
0.82153 


F  Ratio 
6.9601 
Prob  >  F 
0.0001 


Lack  Of  Fit 


Source  DF 

Lack  Of  Fit  3 

Pure  Error  58 

Total  Error  61 


Sum  of  Squares 
2.106703 
48.006410 
50.113114 


Mean  Square  F  Ratio 

0.702234  0.8484 

0.827697  Prob  >  F 

0.4731 
Max  RSq 
0.3422 


Parameter  Estimates 


Term 

Estimate 

Std  Error 

t  Ratio  Prob>|t| 

Intercept 

2.9197364 

0.141338 

20.66  <.0001 

whattypesystemsavionics[1  -0] 

0.967084 

0.482566 

2.00  0.0495 

whattypesystemsdevice[1  -0] 

-1.01028 

0.430037 

-2.35  0.0221 

whattypeenterpriseOES[1  -0] 

1.5048105 

0.654449 

2.30  0.0249 

whatprocessSWEngSuppt[1  -0] 

-0.849094 

0.24403 

-3.48  0.0009 

Effect  T ests 

Source 

Nparm  DF 

Sum  of  Squares 

F  Ratio 

Prob  >  F 

whattypesystemsavionics 

1  1 

3.2994168 

4.0162 

0.0495 

whattypesystemsdevice 

1  1 

4.5341346 

5.5192 

0.0221 

whattypeenterpriseOES 

1  1 

4.3434369 

5.2870 

0.0249 

whatprocessSWEngSuppt 

1  1 

9.9459772 

12.1067 

0.0009 

RlzResponseCust  =  2.92  +  (-.85)proc-SWEngSup  +  (1.50)ent-OES  +  (.97)sys-avia  +  (-l.Ol)sys-dev 


Again,  exact  formula  included  in  dissertation  [11]. 


Given  the  number  of  indicator  variables  of  both  formulas,  there  is  an  extremely  high  confidence  that  these 
modeling  methods  are  exactly  the  same  as  those  practiced  during  the  dissertation. 
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Appendix  B  -  New  Survey  Data  Consequence  Models 


Stepwise  Fit  -  New  Survey  Data  -  Consequences  (Cost) 

Response: 

Column  39 


Stepwise  Regression  Control 


Prob  to  Enter  0.250 

Prob  to  Leave  0.250 

Direction: 

Rules: 


Current  Estimates 

SSE  DFE 

79.357834  39 


MSE  RSquare 

2.0348162  0.3547 


RSquare  Adj 
0.2554 


Cp  AIC 

-13.95063  39.08499 


Lock 

X 


X  Intercept 

whattypesystemsavionics{0-1} 
whattypesystemsembedded{0-1 } 

X  whattypesystemscommunications{0-1 } 

whattypesystemsdevice{0-1} 
whattypeshrinkbusiness{0-1} 
whattypeshrinkutilities{0-1 } 
whattypeshrinkinternet{0-1 } 
whattypecomponentdomain{0-1} 

X  whattypecomponentCASE{0-1} 

whattypecomponentclass{0- 1 } 
whattypecomponentOS{0-1 } 
whattypecomponentdevelopment{0-1} 
whattypeenterpriseacctng{0-1 } 

X  whattypeenterprisemanufact{0-1} 

X  whattypeenterprisepayroll{0-1} 

whattypeenterprisescripting{0-1} 
whatprocessdesign{0- 1 } 
whatprocesstesting{0-1} 
whatprocessmaintenance{0-1 } 
whatprocessreengineering{0-1} 
whatprocessappsuppt{0-1 } 
whatprocesstraining{0-1} 
whatprocessspecification{0-1 } 
whatprocesscoding{0- 1 } 
whatprocessfielding{0-1} 
whatprocessCM{0-1} 
whatprocesstoolsuppt{0-1} 
whatprocessSWEngSuppt{0- 1 } 
whatprocessnone{0-1 } 

X  whatproductscustom{0-1} 

whatproductsCOTS{0-1} 

X  whatproductscommoncust{0-1} 

whatproductsnone{0-1 } 


Step  History 

Step  Parameter 

1  whattypesystemscommunications{0-1 } 

2  whatproductscustom{0-1} 

3  whattypeenterprisepayroll{0-1} 

4  whattypecomponentCASE{0-1} 

5  whatproductscommoncust{0-1} 

6  whattypeenterprisemanufact{0-1} 


Estimate 

2.08493921 

0 

0 

-0.4699771 

0 

0 

0 

0 

0 

1.66493178 

0 

0 

0 

0 

-0.6809201 

0.860886 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0.68908366 

0 

-0.4461527 

0 


nDF  SS 

1  0 

1  1.009938 

1  0.38648 

1  6.932706 

1  0.054901 

1  0.626525 

1  0.078013 

1  2.002562 

1  1.046793 

1  15.79366 

1  1.788671 

1  0.359632 

1  1.501858 

1  1.13712 

1  4.29575 

1  10.52068 

1  1.505615 

1  1.547712 

1  0.103145 

1  0.488836 

1  1.821753 

1  0.055011 

1  0.1246 

1  0.302146 

1  0.018649 

1  1.595572 

1  0.02836 

1  0.211393 

1  0.179373 

1  0.212452 

1  17.4708 

1  0.567121 

1  4.87761 

1  0.098 


"F  Ratio"  "Prob>F" 

0.000  1.0000 

0.490  0.4883 

0.186  0.6687 

3.407  0.0725 

0.026  0.8720 

0.302  0.5856 

0.037  0.8477 

0.984  0.3276 

0.508  0.4804 

7.762  0.0082 

0.876  0.3551 

0.173  0.6798 

0.733  0.3973 

0.552  0.4619 

2.111  0.1542 

5.170  0.0286 

0.735  0.3967 

0.756  0.3901 

0.049  0.8252 

0.236  0.6302 

0.893  0.3507 

0.026  0.8719 

0.060  0.8082 

0.145  0.7053 

0.009  0.9252 

0.780  0.3828 

0.014  0.9078 

0.101  0.7518 

0.086  0.7708 

0.102  0.7512 

8.586  0.0056 

0.274  0.6040 

2.397  0.1296 

0.047  0.8296 


Action 

Entered 

Entered 

Entered 

Entered 

Entered 

Entered 


"Sig  Prob"  Seq  SS 

0.0849  8.110728 

0.0778  8.106118 

0.0658  8.361054 

0.0462  9.19192 

0.1110  5.554858 

0.1542  4.29575 


RSquare  Cp  p 

0.0660  -15.87  2 

0.1319  -15.72  3 

0.1999  -15.62  4 

0.2746  -15.71  5 

0.3198  -14.97  6 

0.3547  -13.95  7 
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Response  Column  39 
Actual  by  Predicted  Plot 
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Column  39  Predicted  P=0.0065  RSq=0.35 
RMSE=1 .4265 


Summary  of  Fit 


RSquare 
RSquare  Adj 
Root  Mean  Square  Error 
Mean  of  Response 
Observations  (or  Sum  Wgts) 

Analysis  of  Variance 


Source  DF 

Model  6 

Error  39 

C.  Total  45 


0.3547 

0.255423 

1.42647 

2.978261 

46 


Sum  of  Squares 
43.62043 
79.35783 
122.97826 


Mean  Square  F  Ratio 

7.27007  3.5728 

2.03482  Prob  >  F 

0.0065 


Lack  Of  Fit 


Source  DF 

Lack  Of  Fit  8 

Pure  Error  31 

Total  Error  39 


Sum  of  Squares 
19.552278 
59.805556 
79.357834 


Mean  Square  F  Ratio 

2.44403  1.2669 

1.92921  Prob  >  F 

0.2960 
Max  RSq 
0.5137 


Parameter  Estimates 

Term 

Intercept 

whattypesystemscommunications[1-0] 
whattypecomponentCASE[1  -0] 
whattypeenterprisemanufact[1  -0] 
whattypeenterprisepayrolljl  -0] 
whatproductscustom[1-0] 
whatproductscommoncust[1-0] 


Estimate 

3.7027908 

0.9399542 

-3.329864 

1.3618402 

-1.721772 

-1.378167 

0.8923054 


Std  Error  t  Ratio  Prob>|t| 
0.413969  8.94  <.0001 

0.509235  1.85  0.0725 

1.19522  -2.79  0.0082 

0.93728  1.45  0.1542 

0.75721  -2.27  0.0286 

0.470336  -2.93  0.0056 

0.576332  1.55  0.1296 


Effect  Tests 

Source 

whattypesystemscommunications 

whattypecomponentCASE 

whattypeenterprisemanufact 

whattypeenterprisepayroll 

whatproductscustom 

whatproductscommoncust 


Nparm 

DF 

Sum  of  Squares 

F  Ratio 

Prob  >  F 

1 

1 

6.932706 

3.4070 

0.0725 

1 

1 

15.793655 

7.7617 

0.0082 

1 

1 

4.295750 

2.1111 

0.1542 

1 

1 

10.520685 

5.1703 

0.0286 

1 

1 

17.470805 

8.5859 

0.0056 

1 

1 

4.877610 

2.3971 

0.1296 

Cost  =  3.70  +  (.94)sys-comm  +  (-3.33)comp-case  +  (1.36)ent-mnft  +  (-1.72)ent-pay  +  (-1.38)prod-cust  +  (.89)prod- 
comcust 
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Stepwise  Fit  -  New  Survey  Data  -  Consequences  (Schedule) 

Response: 

Column  40 


Stepwise  Regression  Control 

Prob  to  Enter  0.250 

Prob  to  Leave  0.250 

Direction: 

Rules: 

Current  Estimates 


SSE 

DFE  MSE  RSquare 

RSquare  Adj 

Cp 

AIC 

39.131272 

36  1.0869798  0.5056 

0.3820 

-6.218063 

12.56091 

Lock 

Entered 

Parameter 

Estimate 

nDF 

SS 

"F  Ratio" 

"Prob>F" 

X 

X 

Intercept 

2.52914565 

1 

0 

0.000 

1.0000 

whattypesystemsavionics{0-1 } 

0 

1 

1.016683 

0.934 

0.3406 

X 

whattypesystemsembedded{0-1 } 

-0.4035016 

1 

1.553509 

1.429 

0.2397 

whattypesystemscommunications{0-1} 

0 

1 

0.176682 

0.159 

0.6927 

X 

whattypesystemsdevice{0- 1 } 

1 .24929933 

1 

16.72195 

15.384 

0.0004 

X 

whattypeshrinkbusiness{0-1} 

-0.5496872 

1 

9.960641 

9.164 

0.0045 

X 

whattypeshrinkutilities{0-1} 

-0.5516462 

1 

2.622341 

2.413 

0.1291 

whattypeshrinkinternet{0-1 } 

0 

1 

1.283646 

1.187 

0.2834 

whattypecomponentdomain{0-1} 

0 

1 

0.00319 

0.003 

0.9577 

whattypecomponentCASE{0-1} 

0 

1 

0.078564 

0.070 

0.7923 

whattypecomponentclass{0- 1 } 

0 

1 

0.013307 

0.012 

0.9137 

whattypecomponentOS{0-1 } 

0 

1 

0.19009 

0.171 

0.6819 

whattypecomponentdevelopment{0-1} 

0 

1 

0.178055 

0.160 

0.6916 

X 

whattypeenterpriseacctng{0-1 } 

0.57666194 

1 

4.99132 

4.592 

0.0390 

whattypeenterprisemanufact{0-1} 

0 

1 

1.116452 

1.028 

0.3176 

whattypeenterprisepayroll{0-1 } 

0 

1 

0.812725 

0.742 

0.3948 

X 

whattypeenterprisescripting{0-1 } 

0.89995918 

1 

5.758236 

5.297 

0.0273 

whatprocessrequirements{0-1} 

0 

1 

0.651667 

0.593 

0.4465 

whatprocessdesign{0-1} 

0 

1 

0.080085 

0.072 

0.7903 

whatprocesstesting{0-1} 

0 

1 

0.199797 

0.180 

0.6743 

X 

whatprocessmaintenance{0-1} 

0.46394251 

1 

6.252855 

5.753 

0.0218 

X 

whatprocessreengineering{0-1 } 

-0.4486158 

1 

5.455645 

5.019 

0.0313 

whatprocessappsuppt{0-1 } 

0 

1 

0.069306 

0.062 

0.8047 

X 

whatprocesstrai  ni  ng{0- 1 } 

-0.6298988 

1 

9.797314 

9.013 

0.0048 

whatprocessspecification{0-1 } 

0 

1 

0.008293 

0.007 

0.9318 

whatprocesscoding{0- 1 } 

0 

1 

0.047741 

0.043 

0.8374 

whatprocessfielding{0-1} 

0 

1 

0.218545 

0.197 

0.6602 

whatprocessCM{0-1} 

0 

1 

0.122813 

0.110 

0.7419 

whatprocesstoolsuppt{0-1} 

0 

1 

0.000385 

0.000 

0.9853 

whatprocessSWEngSuppt{0- 1 } 

0 

1 

0.001822 

0.002 

0.9680 

whatprocessnone{0-1 } 

0 

1 

0.226425 

0.204 

0.6545 

whatproductscustom{0-1 } 

0 

1 

0.905116 

0.829 

0.3689 

whatproductsCOTS{0-1} 

0 

1 

0.33404 

0.301 

0.5865 

whatproductscommoncust{0-1} 

0 

1 

1.153747 

1.063 

0.3095 

whatproductsnone{0-1} 

0 

1 

0.181474 

0.163 

0.6888 

Step  History 

Step 

Parameter 

Action 

"Sig  Prob" 

Seq  SS 

RSquare 

Cp 

P 

1 

whattypeshrinkbusiness{0-1 } 

Entered 

0.0650 

5.959191 

0.0753 

-4.999 

2 

2 

whattypesystemsdevice{0- 1 } 

Entered 

0.0173 

9.126316 

0.1906 

-7.613 

3 

3 

whattypesystemsembedded{0-1 } 

Entered 

0.0788 

4.595922 

0.2487 

-7.936 

4 

4 

whatprocesstraining{0-1 } 

Entered 

0.1613 

2.811707 

0.2842 

-7.357 

5 

5 

whattypeenterprisescripting{0-1} 

Entered 

0.1407 

3.028336 

0.3224 

-6.888 

6 

6 

whattypeenterpriseacctng{0-1 } 

Entered 

0.0990 

3.658833 

0.3687 

-6.738 

7 

7 

whatprocessreengineering{0-1 } 

Entered 

0.0846 

3.810213 

0.4168 

-6.664 

8 

8 

whatprocessmaintenance{0-1} 

Entered 

0.0556 

4.408042 

0.4725 

-6.892 

9 

9 

whattypeshrinkutilities{0-1} 

Entered 

0.1291 

2.622341 

0.5056 

-6.218 

10 

Appendix  B 


Response  Column  40 
Actual  by  Predicted  Plot 


I 

§ 


1  2  3  4  5  6 

Column  40  Predicted  P=0.001 1  RSq=0.51 
RMSE=1.0426 


Summary  of  Fit 


RSquare  0.50562 

RSquare  Adj  0.382025 

Root  Mean  Square  Error  1 .042583 

Mean  of  Response  3.413043 

Observations  (or  Sum  Wgts)  46 


Analysis  of  Variance 


Source  DF 

Model  9 

Error  36 

C.  Total  45 


Sum  of  Squares 
40.020902 
39.131272 
79.152174 


Mean  Square  F  Ratio 

4.44677  4.0909 

1 .08698  Prob  >  F 

0.001 1 


Lack  Of  Fit 

Source  DF  Sum  of  Squares 

Lack  Of  Fit  16  19.881272 

Pure  Error  20  19.250000 

Total  Error  36  39.131272 


Mean  Square  F  Ratio 

1.24258  1.2910 

0.96250  Prob  >  F 

0.2910 
Max  RSq 
0.7568 


Parameter  Estimates 

Term 

Intercept 

whattypesystemsembedded[1  -0] 
whattypesystemsdevice[1  -0] 
whattypeshrinkbusiness[1  -0] 
whattypeshrinkutilities[1  -0] 
whattypeenterpriseacctng[1-0] 
whattypeenterprisescripting[1-0] 
whatprocessmaintenance[1  -0] 
whatprocessreengineering[1  -0] 
whatprocesstraining[1  -0] 


Estimate 

3.1356591 

0.8070031 

-2.498599 

1.0993744 

1.1032924 

-1.153324 

-1.799918 

-0.927885 

0.8972316 

1.2597976 


Std  Error 
0.273249 
0.675039 
0.637036 
0.363172 
0.710324 
0.538213 
0.782022 
0.386871 
0.400491 
0.419622 


t  Ratio 
11.48 
1.20 
-3.92 
3.03 
1.55 
-2.14 
-2.30 
-2.40 
2.24 
3.00 


Prob>|t| 

<.0001 

0.2397 

0.0004 

0.0045 

0.1291 

0.0390 

0.0273 

0.0218 

0.0313 

0.0048 


Effect  T ests 

Source 

whattypesystemsembedded 

whattypesystemsdevice 

whattypeshrinkbusiness 

whattypeshrinkutilities 

whattypeenterpriseacctng 

whattypeenterprisescripting 

whatprocessmaintenance 

whatprocessreengineering 

whatprocesstraining 


Nparm  DF 

1  1 

1  1 

1  1 

1  1 

1  1 

1  1 

1  1 

1  1 

1  1 


Sum  of  Squares 
1.553509 
16.721945 
9.960641 
2.622341 
4.991320 
5.758236 
6.252855 
5.455645 
9.797314 


F  Ratio  Prob  >  F 

1.4292  0.2397 

15.3839  0.0004 

9.1636  0.0045 

2.4125  0.1291 

4.5919  0.0390 

5.2975  0.0273 

5.7525  0.0218 

5.0191  0.0313 

9.0133  0.0048 


Sched  =  3.14  +  (.81)sys-embed  +  (-2.50)sys-dev  +  (I.IO)shrink-bus  +  (I.IO)shrink-util  +  (-1.15)ent-acct  +  (-1.80)ent- 
script  +  (-.93)proc-maint  +  (.90)proc-reeng  +  (1.26)proc-train 
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Stepwise  Fit  -  New  Survey  Data  -  Consequences  (IntelCapital) 

Response: 

Column  40 


Stepwise  Regression  Control 

Prob  to  Enter  0.250 

Prob  to  Leave  0.250 

Direction: 

Rules: 


Current  Estimates 

SSE  DFE 

28.008175  35 


MSE  RSquare 

0.8002336  0.5829 


RSquare  Adj  Cp  AIC 

0.4637  -5.192678  -0.82267 


Lock 

X 


X  Intercept 

whattypesystemsavionics{0-1 } 
whattypesystemsembedded{0-1 } 
whattypesystemscommunications{0-1} 
X  whattypesystemsdevice{0-1} 

whattypeshrinkbusiness{0-1} 
whattypeshrinkutilities{0-1 } 

X  whattypeshrinkinternet{0-1} 

X  whattypecomponentdomain{0-1} 

whattypecomponentCASE{0-1} 
whattypecomponentclass{0- 1 } 

X  whattypecomponentOS{0-1} 

whattypecomponentdevelopment{0-1} 
X  whattypeenterpriseacctng{0-1} 

whattypeenterprisemanufact{0-1 } 
whattypeenterprisepayroll{0-1 } 
whattypeenterprisescripting{0-1} 
whatprocessrequirements{0-1 } 
whatprocessdesign{0- 1 } 
whatprocesstesting{0- 1 } 

X  whatprocessmaintenance{0-1} 

whatprocessreengineering{0-1} 
whatprocessappsuppt{0-1 } 
whatprocesstraining{0-1} 
whatprocessspecification{0-1 } 
whatprocesscoding{0- 1 } 

X  whatprocessfielding{0-1} 

X  whatprocessCM{0-1} 

X  whatprocesstoolsuppt{0-1} 

X  whatprocessSWEngSuppt{0-1} 

whatprocessnone{0-1 } 
whatproductscustom{0-1 } 
whatproductsCOTS{0-1} 
whatproductscommoncust{0-1} 
whatproductsnone{0-1} 


Step  History 

Step  Parameter 

1  whatprocessSWEngSuppt{0-1} 

2  whattypeshrinkinternet{0-1} 

3  whattypecomponentdomain{0-1} 

4  whatprocesstraining{0-1} 

5  whattypecomponentOS{0-1} 

6  whatprocessCM{0-1} 

7  whatprocesstoolsuppt{0-1} 

8  whatprocessmaintenance{0-1} 

9  whattypesystemsdevice{0-1} 

10  whattypeenterpriseacctng{0-1} 

1 1  whatprocesstraining{0-1 } 

12  whatprocessfielding{0-1) 


Action 

Entered 

Entered 

Entered 

Entered 

Entered 

Entered 

Entered 

Entered 

Entered 

Entered 

Removed 

Entered 


Estimate  nDF 

5.3776271  1 

0  1 

0  1 

0  1 

-0.3894834  1 

0  1 

0  1 

-0.6819844  1 

-0.4996501  1 

0  1 

0  1 

0.72092249  1 

0  1 

-0.4207597  1 

0  1 

0  1 

0  1 

0  1 

0  1 

0  1 

-0.40975  1 

0  1 

0  1 

0  1 

0  1 

0  1 

0.38457251  1 

0.73327355  1 

-0.7504809  1 

-0.6836766  1 

0  1 

0  1 

0  1 

0  1 

0  1 


SS 

0 

0.095315 

0.923402 

0.302138 

I. 594557 
0.243258 
0.943237 

II. 70104 
2.317897 
0.145889 
0.157367 
5.029578 
0.068966 
2.633816 
0.187128 
0.410308 

0.02766 

0.489014 

0.032983 

0.425346 

5.070822 

0.00577 

0.146864 

0.368537 

0.748333 

0.133412 

1.115911 

7.324149 

6.372963 

8.930003 

0.333058 

0.415287 

0.067629 

0.331416 

0.020338 


"Sig  Prob" 
0.0059 
0.0182 
0.0155 
0.1331 
0.0538 
0.1925 
0.1247 
0.2109 
0.1538 
0.2097 
0.4955 
0.2456 


Seq  SS 
10.71914 
6.935211 
6.521159 
2.327595 
3.652109 
1.596481 
2.156643 
1.395474 
1.772972 
1 .340845 
0.389541 
1.115911 


RSquare 

0.1596 

0.2629 

0.3600 

0.3947 

0.4491 

0.4728 

0.5049 

0.5257 

0.5521 

0.5721 

0.5663 

0.5829 


"F  Ratio"  "Prob>F" 

0.000  1.0000 

0.116  0.7354 

1.159  0.2892 

0.371  0.5466 

1.993  0.1669 

0.298  0.5888 

1.185  0.2840 

14.622  0.0005 

2.897  0.0976 

0.178  0.6757 

0.192  0.6639 

6.285  0.0170 

0.084  0.7738 

3.291  0.0782 

0.229  0.6356 

0.505  0.4819 

0.034  0.8556 

0.604  0.4424 

0.040  0.8425 

0.524  0.4740 

6.337  0.0166 

0.007  0.9338 

0.179  0.6747 

0.453  0.5053 

0.933  0.3408 

0.163  0.6892 

1.394  0.2456 

9.153  0.0046 

7.964  0.0078 

11.159  0.0020 

0.409  0.5267 

0.512  0.4793 

0.082  0.7760 

0.407  0.5277 

0.025  0.8760 


Cp  p 

-4.106  2 

-6.763  3 

-9.141  4 

-8.704  5 

-9.157  6 

-8.229  7 

-7.677  8 

-6.614  9 

-5.805  10 

-4.705  1 1 

-6.443  10 

-5.193  11 


Appendix  B 


Response  Column  40 
Actual  by  Predicted  Plot 


RMSE=0.8946 

Summary  of  Fit 


RSquare 
RSquare  Adj 
Root  Mean  Square  Error 
Mean  of  Response 
Observations  (or  Sum  Wgts) 
Analysis  of  Variance 
Source  DF 

Model  10 

Error  35 

C.  Total  45 

Lack  Of  Fit 

Source  DF 

Lack  Of  Fit  17 

Pure  Error  18 

Total  Error  35 


Parameter  Estimates 

Term 

Intercept 

whattypesystemsdevice[1  -0] 
whattypeshrinkinternet[1  -0] 
whattypecomponentdomain[1  -0] 
whattypecomponentOS[1  -0] 
whattypeenterpriseacctngjl  -0] 
whatprocessmaintenance[1  -0] 
whatprocessfieldingjl  -0] 
whatprocessCMjl  -0] 
whatprocesstoolsuppt[1  -0] 
whatprocessSWEngSuppt[1  -0] 


0.582915 

0.463748 

0.894558 

4.413043 

46 


F  Ratio 
4.8916 
Prob  >  F 
0.0002 


F  Ratio 
0.7372 
Prob  >  F 
0.7330 
Max  RSq 
0.7541 


Sum  of  Squares 
39.143999 
28.008175 
67.152174 


Sum  of  Squares 
11.496054 
16.512121 
28.008175 


Mean  Square 
3.91440 
0.80023 


Mean  Square 
0.676238 
0.917340 


Estimate  Std  Error  t  Ratio 
3.3806104  0.228154  14.82 

0.7789669  0.551833  1.41 

1 .3639689  0.356698  3.82 

0.9993003  0.587161  1.70 

-1.441845  0.575124  -2.51 

0.8415194  0.463852  1.81 

0.8195  0.32555  2.52 

-0.769145  0.651331  -1.18 

-1.466547  0.484759  -3.03 

1.5009619  0.531872  2.82 

1.3673532  0.40932  3.34 


Prob>|t| 

<.0001 

0.1669 

0.0005 

0.0976 

0.0170 

0.0782 

0.0166 

0.2456 

0.0046 

0.0078 

0.0020 


Effect  Tests 

Source 

whattypesystemsdevice 

whattypeshrinkinternet 

whattypecomponentdomain 

whattypecomponentOS 

whattypeenterpriseacctng 

whatprocessmaintenance 

whatprocessfielding 

whatprocessCM 

whatprocesstoolsuppt 

whatprocessSWEngSuppt 


Nparm  DF 

1  1 

1  1 

1  1 

1  1 

1  1 

1  1 

1  1 

1  1 

1  1 

1  1 


Sum  of  Squares 
1.594557 
11.701037 
2.317897 
5.029578 
2.633816 
5.070822 
1.115911 
7.324149 
6.372963 
8.930003 


F  Ratio  Prob  >  F 

1.9926  0.1669 

14.6220  0.0005 

2.8965  0.0976 

6.2851  0.0170 

3.2913  0.0782 

6.3367  0.0166 

1.3945  0.2456 

9.1525  0.0046 

7.9639  0.0078 

11.1592  0.0020 


IntelCap  =  3.38  +  (.78)sys-dev  +  (1.36)shrink-int  +  (I.OO)comp-domain  +  (-1 .44)comp-os  +  (.84)ent-acct  +  (.82)proc-maint 
+  (-.77)proc-field  +  (-1 .47)proc-cm  +  (1.50)proc-toolsup  +  (1.37)proc-swengsup 
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Stepwise  Fit  -  New  Survey  Data  -  Consequences  (SchedFlex) 

Response: 

Column  40 


Stepwise  Regression  Control 

Prob  to  Enter  0.250 

Prob  to  Leave  0.250 

Direction: 

Rules: 


Current  Estimates 

SSE  DFE 

30.045838  32 


MSE  RSquare 

0.9389324  0.7399 


RSquare  Adj  Cp  AIC 

0.6342  -2.396465  8.407807 


Lock 

X 


X  Intercept 

whattypesystemsavionics{0-1 } 

X  whattypesystemsembedded{0-1} 

whattypesystemscommunications{0-1} 

whattypesystemsdevice{0-1} 

X  whattypeshrinkbusiness{0-1} 

whattypeshrinkutilities{0-1 } 
whattypeshrinkinternet{0-1 } 
whattypecomponentdomain{0-1} 

X  whattypecomponentCASE{0-1} 

X  whattypecomponentclass{0-1} 

whattypecomponentOS{0-1 } 

X  whattypecomponentdevelopment{0- 1 } 

X  whattypeenterpriseacctng{0-1} 

whattypeenterprisemanufact{0-1 } 

X  whattypeenterprisepayroll{0-1} 

whattypeenterprisescripting{0-1} 
whatprocessrequirements{0-1 } 

X  whatprocessdesign{0-1} 

X  whatprocesstesting{0-1} 

whatprocessmaintenance{0-1} 
whatprocessreengineering{0-1} 
whatprocessappsuppt{0-1 } 
whatprocesstraining{0-1} 
whatprocessspecification{0-1 } 

X  whatprocesscoding{0-1} 

whatprocessfielding{0-1} 

X  whatprocessCM{0-1} 

whatprocesstoolsuppt{0-1} 

X  whatprocessSWEngSuppt{0-1} 

whatprocessnone{0-1 } 
whatproductscustom{0-1 } 
whatproductsCOTS{0-1} 

X  whatproductscommoncust{0-1} 

whatproductsnone{0-1} 


Step  History 

Step  Parameter 

1  whattypesystemsembedded{0-1} 

2  whattypeshrinkbusiness{0-1} 

3  whatproductsnone{0-1} 

4  whattypecomponentclass{0-1} 

5  whatproductscommoncust{0-1} 

6  whatprocessnone{0-1} 

7  whatprocesstesting{0-1} 

8  whatprocessCM{0-1} 

9  whatprocessrequirements{0-1} 

10  whattypecomponentCASE{0-1} 

1 1  whatprocessnone{0-1 } 

12  whatprocessSWEngSuppt{0-1} 

13  whatproductsnone{0-1} 

14  whattypeenterprisepayroll{0-1} 

15  whatprocessrequirements{0-1} 

16  whattypeenterpriseacctng{0-1} 

1 7  whattypecomponentdevelopment{0- 1 } 

18  whatprocessdesign{0-1} 

19  whatprocesscoding{0-1} 


Estimate  nDF 

-0.3545249  1 

0  1 

1.26487859  1 

0  1 

0  1 

0.54952738  1 

0  1 

0  1 

0  1 

2.50028057  1 

1 .47000467  1 

0  1 

0.65905834  1 

-0.9622171  1 

0  1 

0.59615824  1 

0  1 

0  1 

0.62655563  1 

-1.0971388  1 

0  1 

0  1 

0  1 

0  1 

0  1 

-0.2741587  1 

0  1 

1.50517411  1 

0  1 

-0.9210343  1 

0  1 

0  1 

0  1 

-0.8867386  1 

0  1 


SS 

0 

0.774739 

16.11252 

0.14589 

0.001872 

11.81094 

0.078544 

0.407695 

0.068426 

23.25663 

15.04065 

0.492675 

7.304894 

9.845101 

0.347059 

4.067231 

0.649942 

0.053362 

5.195615 

17.88281 

0.238712 

0.138292 

0.009611 

0.445768 

0.000094 

1.452279 

0.972452 

26.74184 

0.022699 

10.97408 

0.043465 

0.146213 

0.612217 

14.22332 

0.007464 


"F  Ratio" 
0.000 
0.820 
17.160 
0.151 
0.002 
12.579 
0.081 
0.426 
0.071 
24.769 
16.019 
0.517 
7.780 
10.485 
0.362 
4.332 
0.685 
0.055 
5.534 
19.046 
0.248 
0.143 
0.010 
0.467 
0.000 
1.547 
1.037 
28.481 
0.023 
11.688 
0.045 
0.152 
0.645 
15.148 
0.008 


'Prob>F" 

1.0000 

0.3720 

0.0002 

0.7000 

0.9652 

0.0012 

0.7775 

0.5186 

0.7920 

0.0000 

0.0003 

0.4776 

0.0088 

0.0028 

0.5516 

0.0455 

0.4141 

0.8159 

0.0250 

0.0001 

0.6218 

0.7076 

0.9213 

0.4995 

0.9922 

0.2227 

0.3164 

0.0000 

0.8793 

0.0017 

0.8336 

0.6997 

0.4281 

0.0005 

0.9306 


Action 

Entered 

Entered 

Entered 

Entered 

Entered 

Entered 

Entered 

Entered 

Entered 

Entered 

Removed 

Entered 

Removed 

Entered 

Removed 

Entered 

Entered 

Entered 

Entered 


'Sig  Prob"  Seq  SS 

0.0137  15.06589 

0.0150  13.05319 

0.0123  12.25009 

0.0917  5.093144 

0.1199  4.159325 

0.1981  2.773217 

0.1439  3.493746 

0.0431  6.321743 

0.1824  2.60234 

0.1993  2.363301 

0.4284  0.886522 

0.1359  3.071432 

0.4113  0.911691 

0.0511  4.913742 

0.3929  0.900881 

0.1181  2.941165 

0.0441  4.568681 

0.0479  4.029973 

0.2227  1.452279 


RSquare  Cp  p 

0.1304  10.158  2 

0.2435  5.379  3 

0.3495  1.0173  4 

0.3936  0.3723  5 

0.4296  0.2122  6 

0.4536  0.772  7 

0.4839  0.9577  8 

0.5386  -0.325  9 

0.5611  0.3232  10 

0.5816  1.0958  11 

0.5739  -0.444  10 

0.6005  -0.039  11 

0.5926  -1.565  10 

0.6352  -2.117  11 

0.6274  -3.649  10 

0.6528  -3.177  11 

0.6924  -3.549  12 

0.7273  -3.642  13 

0.7399  -2.396  14 
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Response  Column  40 
Actual  by  Predicted  Plot 


RMSE=0.969 

Summary  of  Fit 


RSquare 
RSquare  Adj 
Root  Mean  Square  Error 
Mean  of  Response 
Observations  (or  Sum  Wgts) 
Analysis  of  Variance 
Source  DF 

Model  13 

Error  32 

C.  Total  45 

Lack  Of  Fit 

Source  DF 

Lack  Of  Fit  19 

Pure  Error  13 

Total  Error  32 


0.739863 

0.634182 

0.968985 

4.5 

46 


Sum  of  Squares 
85.45416 
30.04584 
115.50000 

Sum  of  Squares 
25.379171 
4.666667 
30.045838 


Mean  Square 
6.57340 
0.93893 


Mean  Square 
1.33575 
0.35897 


Parameter  Estimates 

Term 

Intercept 

whattypesystemsembedded[1  -0] 
whattypeshrinkbusiness[1  -0] 
whattypecomponentCASE[1  -0] 
whattypecomponentclass[1  -0] 
whattypecomponentdevelopment[1-0] 
whattypeenterpriseacctng[1  -0] 
whattypeenterprisepayroll[1  -0] 
whatprocessdesign[1  -0] 
whatprocesstesting[1  -0] 
whatprocesscoding[1  -0] 
whatprocessCM[1  -0] 
whatprocessSWEngSuppt[1  -0] 
whatproductscommoncustfl  -0] 
Effect  Tests 
Source 

whattypesystemsembedded 

whattypeshrinkbusiness 

whattypecomponentCASE 

whattypecomponentclass 

whattypecomponentdevelopment 

whattypeenterpriseacctng 

whattypeenterprisepayroll 

whatprocessdesign 

whatprocesstesting 

whatprocesscoding 

whatprocessCM 

whatprocessSWEngSuppt 

whatproductscommoncust 


Estimate 

4.6758251 

-2.529757 

-1.099055 

-5.000561 

-2.940009 

-1.318117 

1.9244342 

-1.192316 

-1.253111 

2.1942777 

0.5483174 

-3.010348 

1.8420686 

1.7734772 


Nparm  DF 

1  1 

1  1 

1  1 

1  1 

1  1 

1  1 

1  1 

1  1 

1  1 

1  1 

1  1 

1  1 

1  1 


F  Ratio 
7.0009 
Prob  >  F 
<.0001 


F  Ratio 
3.7210 
Prob  >  F 
0.0096 
Max  RSq 
0.9596 


Std  Error 
0.357505 
0.610681 
0.309881 
1.00476 
0.734569 
0.472568 
0.594306 
0.572874 
0.532707 
0.502795 
0.440884 
0.564077 
0.538814 
0.455661 


Sum  of  Squares 
16.112522 
11.810936 
23.256633 
15.040651 
7.304894 
9.845101 
4.067231 
5.195615 
17.882806 
1 .452279 
26.741840 
10.974078 
14.223322 


t  Ratio  Prob>|t| 

13.08  <.0001 

-4.14  0.0002 

-3.55  0.0012 

-4.98  <.0001 

-4.00  0.0003 

-2.79  0.0088 

3.24  0.0028 

-2.08  0.0455 

-2.35  0.0250 

4.36  0.0001 

1 .24  0.2227 

-5.34  <.0001 

3.42  0.0017 

3.89  0.0005 


F  Ratio  Prob  >  F 

17.1605  0.0002 

12.5791  0.0012 

24.7692  <.0001 

16.0189  0.0003 

7.7800  0.0088 

10.4854  0.0028 

4.3318  0.0455 

5.5335  0.0250 

19.0459  0.0001 

1.5467  0.2227 

28.4811  <.0001 

11.6878  0.0017 

15.1484  0.0005 


SchedFlex  =  4.68  +  sys-embed(-2.53)  +  shrink-bus(-I.IO)  +  comp-CASE(-5.00)  +  comp-class(-2.94)  +  comp-dev(-1 .32)  + 
ent-acct(1.92)  +  ent-pay(-1.19)  +  proc-des(-1 .25)  +  proc-test(2.19)  +  proc-coding(0.55)  +  proc-CM(-3.01)  +  proc- 
SWEngSup(1.84)  +  prod-comcust(1.77) 
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Stepwise  Fit  -  New  Survey  Data  -  Consequences  (AdminOverhead) 

Response: 

Column  59 


Stepwise  Regression  Control 

Prob  to  Enter  0.250 

Prob  to  Leave  0.250 

Direction: 

Rules: 


Current  Estimates 

SSE  DFE 

66.493031  36 


MSE  RSquare 

1.8470286  0.5283 


RSquare  Adj 
0.4103 


Cp  AIC 

-8.477527  36.94896 


Lock  Entered  Parameter 

X  X  Intercept 

whattypesystemsavionics{0-1 } 
whattypesystemsembedded{0-1 } 
whattypesystemscommunications{0-1 
whattypesystemsdevice{0- 1 } 
whattypeshrinkbusiness{0-1} 
whattypeshrinkutilities{0-1 } 

X  whattypeshrinkinternet{0-1} 

whattypecomponentdomain{0-1} 

whattypecomponentCASE{0-1} 

X  whattypecomponentclass{0-1} 

X  whattypecomponentOS{0-1} 

whattypecomponentdevelopment{0-1 
whattypeenterpriseacctng{0-1 } 
whattypeenterprisemanufact{0-1} 
whattypeenterprisepayroll{0-1 } 
whattypeenterprisescripting{0- 1 } 

X  whatprocessrequirements{0-1} 

whatprocessdesign{0- 1 } 

X  whatprocesstesting{0-1} 

whatprocessmaintenance{0-1} 
whatprocessreengineering{0-1 } 
whatprocessappsuppt{0-1 } 
whatprocesstraining{0-1} 
whatprocessspecification{0-1 } 
whatprocesscoding{0- 1 } 
whatprocessfielding{0-1} 
whatprocessCM{0-1} 

X  whatprocesstoolsuppt{0-1} 

X  whatprocessSWEngSuppt{0-1} 

whatprocessnone{0-1 } 
whatproductscustom{0-1 } 
whatproductsCOTS{0-1} 

X  whatproductscommoncust{0-1} 

whatproductsnone{0-1} 


Step  History 

Step  Parameter 

1  whatproductsnone{0-1} 

2  whattypecomponentclass{0-1} 

3  whattypecomponentOS{0-1} 

4  whatprocesstesting{0-1} 

5  whatprocessrequirements{0-1} 

6  whatprocessSWEngSuppt{0-1} 

7  whatprocesstoolsuppt{0-1} 

8  whattypeshrinkinternet{0-1} 

9  whatproductscommoncust{0-1} 


Action 

Entered 

Entered 

Entered 

Entered 

Entered 

Entered 

Entered 

Entered 

Entered 


Estimate  nDF 

1.72251624  1 

0  1 

0  1 

0  1 

0  1 

0  1 

0  1 

-0.3887688  1 

0  1 

0  1 

1.10308015  1 

0.92052294  1 

0  1 

0  1 

0  1 

0  1 

0  1 

-0.6649058  1 

0  1 

0.6084427  1 

0  1 

0  1 

0  1 

0  1 

0  1 

0  1 

0  1 

0  1 

-0.7607488  1 

0.8481211  1 

0  1 

0  1 

0  1 

-0.3922808  1 

1.2731605  1 


"Sig  Prob" 
0.0058 
0.0405 
0.1290 
0.1492 
0.0503 
0.1232 
0.0382 
0.1641 
0.2087 


Seq  SS 
22.63457 
11.12195 
5.790476 
5.077765 
8.90479 
5.230746 
8.891698 
3.785499 
3.025995 


SS 

0 

0.658375 

0.243243 

0.94832 

1.277188 

0.027209 

1.785783 

3.906444 

0.160176 

I. 715916 
11.313 

8.508319 

0.009701 

0.339783 

0.124606 

0.991785 

0.038605 

10.37938 

0.747069 

II. 59251 
0.02869 

1.031179 

0.466099 

0.022942 

0.379908 

0.985615 

0.78763 

0.313446 

10.2072 

15.13001 

0.178195 

0.680208 

0.001691 

3.025995 

23.11573 


RSquare 

0.1606 

0.2395 

0.2806 

0.3166 

0.3798 

0.4169 

0.4799 

0.5068 

0.5283 


"F  Ratio" 
0.000 
0.350 
0.129 
0.506 
0.685 
0.014 
0.966 
2.115 
0.085 
0.927 
6.125 
4.606 
0.005 
0.180 
0.066 
0.530 
0.020 
5.620 
0.398 
6.276 
0.015 
0.551 
0.247 
0.012 
0.201 
0.527 
0.420 
0.166 
5.526 
8.192 
0.094 
0.362 
0.001 
1.638 
12.515 


Cp 

-10.82 

-11.75 

-11.28 

-10.61 

-10.96 

-10.34 

-10.68 

-9.68 

-8.478 


"Prob>F" 

1.0000 

0.5579 

0.7221 

0.4814 

0.4133 

0.9054 

0.3324 

0.1545 

0.7730 

0.3422 

0.0182 

0.0387 

0.9434 

0.6742 

0.7992 

0.4715 

0.8874 

0.0232 

0.5324 

0.0169 

0.9029 

0.4627 

0.6223 

0.9131 

0.6566 

0.4729 

0.5214 

0.6864 

0.0243 

0.0070 

0.7609 

0.5514 

0.9764 

0.2087 

0.0011 

P 

2 

3 

4 

5 

6 

7 

8 

9 

10 
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Response  Column  59 
Actual  by  Predicted  Plot 


Column  59  Predicted  P=0.0005  RSq=0.53 
RMSE=1.3591 


Summary  of  Fit 


RSquare  0.528273 

RSquare  Adj  0.410341 

Root  Mean  Square  Error  1 .359054 

Mean  of  Response  3.608696 

Observations  (or  Sum  Wgts)  46 


Analysis  of  Variance 

Source  DF  Sum  of  Squares 

Model  9  74.46349 

Error  36  66.49303 

C.  Total  45  140.95652 


Mean  Square  F  Ratio 

8.27372  4.4795 

1 .84703  Prob  >  F 

0.0005 


Lack  Of  Fit 

Source  DF  Sum  of  Squares 

Lack  Of  Fit  17  39.147793 

Pure  Error  19  27.345238 

Total  Error  36  66.493031 


Mean  Square  F  Ratio 

2.30281  1.6000 

1.43922  Prob  >  F 

0.1610 
Max  RSq 
0.8060 


Parameter  Estimates 

Term 

Intercept 

whattypeshrinkinternet[1  -0] 
whattypecomponentclass[1  -0] 
whattypecomponentOS[1  -0] 
whatprocessrequirements[1  -0] 
whatprocesstesting[1  -0] 
whatprocesstoolsuppt[1  -0] 
whatprocessSWEngSuppt[1  -0] 
whatproductscommoncust[1-0] 
whatproductsnone[1-0] 


Estimate 

4.2691395 

0.7775376 

-2.20616 

-1.841046 

1.3298116 

-1.216885 

1.5214975 

-1.696242 

0.7845615 

-2.546321 


Std  Error 
0.425788 
0.534647 
0.891425 
0.857787 
0.560972 
0.485733 
0.647224 
0.592659 
0.612957 
0.719774 


t  Ratio 
10.03 
1.45 
-2.47 
-2.15 
2.37 
-2.51 
2.35 
-2.86 
1.28 
-3.54 


Prob>|t| 

<.0001 

0.1545 

0.0182 

0.0387 

0.0232 

0.0169 

0.0243 

0.0070 

0.2087 

0.0011 


Effect  T ests 

Source 

whattypeshrinkinternet 

whattypecomponentclass 

whattypecomponentOS 

whatprocessrequirements 

whatprocesstesting 

whatprocesstoolsuppt 

whatprocessSWEngSuppt 

whatproductscommoncust 

whatproductsnone 


Nparm  DF 

1  1 

1  1 

1  1 

1  1 

1  1 

1  1 

1  1 

1  1 

1  1 


Sum  of  Squares 
3.906444 
11.312998 
8.508319 
10.379384 
11.592515 
10.207198 
15.130015 
3.025995 
23.115732 


F  Ratio  Prob  >  F 

2.1150  0.1545 

6.1250  0.0182 

4.6065  0.0387 

5.6195  0.0232 

6.2763  0.0169 

5.5263  0.0243 

8.1915  0.0070 

1 .6383  0.2087 

12.5151  0.0011 


AdminOverhead  =  4.27  +  shrink-int(0.78)  +  comp-class(-2.21)  +  comp-OS(-1 .84)  +  proc-req(1.33)  +  proc-test(-1 .22)  + 
proc-toolsup(1.52)  +  proc-SWEngSup(-1 .70)  +  prod-comcust(0.78)  +  prod-none(-2.55) 
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Stepwise  Fit  -  New  Survey  Data  -  Consequences  (ControlProcess) 

Response: 

Column  59 


Stepwise  Regression  Control 

Prob  to  Enter  0.250 

Prob  to  Leave  0.250 

Direction: 

Rules: 


Current  Estimates 

SSE  DFE 

40.357179  33 


MSE  RSquare 

1.2229448  0.6331 


RSquare  Adj 
0.4997 


Cp  AIC 

-0.041754  19.97988 


Lock  Entered  Parameter 

X  X  Intercept 

X  whattypesystemsavionics{0-1} 

whattypesystemsembedded{0-1 } 
whattypesystemscommunications{0-1} 
X  whattypesystemsdevice{0-1} 

whattypeshrinkbusiness{0-1} 

X  whattypeshrinkutilities{0-1} 

whattypeshrinkinternet{0-1 } 
whattypecomponentdomain{0-1} 
whattypecomponentCASE{0-1} 
whattypecomponentclass{0- 1 } 

X  whattypecomponentOS{0-1} 

X  whattypecomponentdevelopment{0-1} 
whattypeenterpriseacctng{0-1 } 

X  whattypeenterprisemanufact{0-1} 

X  whattypeenterprisepayroll{0-1} 

whattypeenterprisescripting{0-1} 
whatprocessrequirements{0-1 } 
whatprocessdesign{0- 1 } 
whatprocesstesting{0- 1 } 
whatprocessmaintenance{0-1} 
whatprocessreengineering{0-1 } 

X  whatprocessappsuppt{0-1} 

whatprocesstraining{0-1} 
whatprocessspecification{0-1 } 

X  whatprocesscoding{0-1} 

whatprocessfielding{0-1} 
whatprocessCM{0-1 } 
whatprocesstoolsuppt{0-1} 

X  whatprocessSWEngSuppt{0-1} 

whatprocessnone{0-1 } 
whatproductscustom{0-1 } 

X  whatproductsCOTS{0-1} 

whatproductscommoncust{0-1} 

X  whatproductsnone{0-1} 

Step  History 

Step  Parameter 

1  whattypeenterpriseacctng{0-1} 

2  whattypesystemsdevice{0-1} 

3  whattypesystemsavionics{0-1} 

4  whatprocessappsuppt{0-1} 

5  whattypecomponentdevelopment{0-1} 

6  whatproductsnone{0-1} 

7  whattypeenterprisemanufact{0-1} 

8  whattypeenterpriseacctng{0-1} 

9  whatprocessdesign{0-1} 

1 0  whattypeenterprisepayroll{0-1 } 

11  whatprocessSWEngSuppt{0-1} 

12  whattypeshrinkutilities{0-1} 

13  whatprocesscoding{0-1} 

14  whattypecomponentOS{0-1} 

15  whatproductsCOTS{0-1} 

16  whatprocessdesign{0-1} 


Estimate  nDF 

4.62803471  1 

-1.0541902  1 

0  1 

0  1 

-0.8931264  1 

0  1 

0.74174585  1 

0  1 

0  1 

0  1 

0  1 

0.74050624  1 

-0.8331768  1 

0  1 

0.50193772  1 

0.51834656  1 

0  1 

0  1 

0  1 

0  1 

0  1 

0  1 

-0.5635149  1 

0  1 

0  1 

0.57219524  1 

0  1 

0  1 

0  1 

-0.5495356  1 

0  1 

0  1 

-0.4300807  1 

0  1 

0.58467308  1 


SS 

0 

7.475665 

0.006575 

0.982319 

9.175545 

0.612079 

4.888518 

0.951513 

0.002036 

0.04261 

0.345931 

4.419838 

16.06059 

0.54165 

2.415963 

3.146373 

0.125498 

0.116795 

0.542842 

0.12514 

0.199997 

0.149328 

9.294997 

0.176959 

0.515109 

7.618211 

0.331069 

0.008992 

0.119657 

7.75271 

0.371568 

1.425588 

3.895401 

0.000772 

4.759368 


"F  Ratio" 
0.000 
6.113 
0.005 
0.798 
7.503 
0.493 
3.997 
0.773 
0.002 
0.034 
0.277 
3.614 
13.133 
0.435 
1.976 
2.573 
0.100 
0.093 
0.436 
0.100 
0.159 
0.119 
7.601 
0.141 
0.414 
6.229 
0.265 
0.007 
0.095 
6.339 
0.297 
1.172 
3.185 
0.001 
3.892 


'Prob>F" 

1.0000 

0.0187 

0.9429 

0.3783 

0.0099 

0.4878 

0.0539 

0.3859 

0.9682 

0.8552 

0.6025 

0.0661 

0.0010 

0.5141 

0.1692 

0.1182 

0.7541 

0.7625 

0.5136 

0.7544 

0.6924 

0.7325 

0.0094 

0.7098 

0.5247 

0.0177 

0.6105 

0.9332 

0.7597 

0.0168 

0.5893 

0.2871 

0.0835 

0.9804 

0.0570 


Action 

Entered 

Entered 

Entered 

Entered 

Entered 

Entered 

Entered 

Removed 

Entered 

Entered 

Entered 

Entered 

Entered 

Entered 

Entered 

Removed 


'Sig  Prob"  Seq  SS 

0.0136  14.36098 

0.0880  6.330916 

0.0649  7.03668 

0.0341  8.631018 

0.0120  10.86669 

0.1698  2.997959 

0.1353  3.451219 

0.2866  1.731321 

0.1225  3.577699 

0.1335  3.257695 

0.1407  3.036362 

0.2302  1.968589 

0.2378  1.88283 

0.1842  2.341313 

0.1953  2.177043 

0.5136  0.542842 


RSquare  Cp  p 

0.1306  5.2973  2 

0.1881  4.1664  3 

0.2521  2.6865  4 

0.3305  0.4181  5 

0.4293  -2.956  6 

0.4566  -2.438  7 

0.4880  -2.145  8 

0.4722  -3.289  7 

0.5047  -3.058  8 

0.5344  -2.669  9 

0.5620  -2.171  10 

0.5799  -1.145  11 

0.5970  -0.076  12 

0.6183  0.7664  13 

0.6381  1.6898  14 

0.6331  -0.042  13 
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Response  Column  59 

Actual  by  Predicted  Plot 
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Column  59  Predicted  P=0.0002  RSq=0.63 
RMSE=1 .1059 


Summary  of  Fit 


RSquare 
RSquare  Adj 
Root  Mean  Square  Error 
Mean  of  Response 
Observations  (or  Sum  Wgts) 
Analysis  of  Variance 
Source  DF 

Model  12 

Error  33 

C.  Total  45 

Lack  Of  Fit 

Source  DF 

Lack  Of  Fit  19 

Pure  Error  14 

Total  Error  33 


0.633117 

0.499704 

1.105868 

4 

46 


Sum  of  Squares 
69.64282 
40.35718 
110.00000 


Sum  of  Squares 
16.523845 
23.833333 
40.357179 


Mean  Square 
5.80357 
1 .22294 


Mean  Square 
0.86968 
1.70238 


Parameter  Estimates 

Term 

Intercept 

whattypesystemsavionics[1  -0] 
whattypesystemsdevice[1  -0] 
whattypeshrinkutilities[1  -0] 
whattypecomponentOS[1  -0] 
whattypecomponentdevelopment[1-0] 
whattypeenterprisemanufact[1  -0] 
whattypeenterprisepayrolljl  -0] 
whatprocessappsuppt[1-0] 
whatprocesscoding[1  -0] 
whatprocessSWEngSuppt[1  -0] 
whatproductsCOTS[1-0] 
whatproductsnone[1-0] 

Effect  Tests 

Source 

whattypesystemsavionics 

whattypesystemsdevice 

whattypeshrinkutilities 

whattypecomponentOS 

whattypecomponentdevelopment 

whattypeenterprisemanufact 

whattypeenterprisepayroll 

whatprocessappsuppt 

whatprocesscoding 

whatprocessSWEngSuppt 

whatproductsCOTS 

whatprod  uctsnone 


Estimate 

3.9638148 

2.1083804 

1.7862529 

-1.483492 

-1.481012 

1.6663535 

-1.003875 

-1.036693 

1.1270298 

-1.14439 

1.0990712 

0.8601614 

-1.169346 


Nparm  DF 

1  1 

1  1 

1  1 

1  1 

1  1 

1  1 

1  1 

1  1 

1  1 

1  1 

1  1 

1  1 


F  Ratio 
4.7456 
Prob  >  F 
0.0002 


F  Ratio 
0.5109 
Prob  >  F 
0.9136 
Max  RSq 
0.7833 


Std  Error 
0.482208 
0.852761 
0.652124 
0.741993 
0.779039 
0.459822 
0.71423 
0.646321 
0.408803 
0.458512 
0.436518 
0.481956 
0.59275 


Sum  of  Squares 
7.475665 
9.175545 
4.888518 
4.419838 
16.060592 
2.415963 
3.146373 
9.294997 
7.618211 
7.752710 
3.895401 
4.759368 


t  Ratio  Prob>|t| 

8.22  <.0001 

2.47  0.0187 

2.74  0.0099 

-2.00  0.0539 

-1.90  0.0661 

3.62  0.0010 

-1.41  0.1692 

-1.60  0.1182 

2.76  0.0094 

-2.50  0.0177 

2.52  0.0168 

1.78  0.0835 

-1.97  0.0570 


F  Ratio  Prob  >  F 

6.1128  0.0187 

7.5028  0.0099 

3.9973  0.0539 

3.6141  0.0661 

13.1327  0.0010 

1.9755  0.1692 

2.5728  0.1182 

7.6005  0.0094 

6.2294  0.0177 

6.3394  0.0168 

3.1853  0.0835 

3.8917  0.0570 


ControlProcess  =  3.96  +  sys-avia(2.11)  +  sys-dev(1.79)  +  shrink-util(-1 .48)  +  comp-OS(-1 .48)  +  comp-dev(1.67)  +  ent- 
mnft(-I.OO)  +  ent-pay(-1 .04)  +  proc-appsup(1.13)  +  proc-coding(-1.14)  +  proc-SWEngSup(I.IO)  +  prod-COTS(0.86)  + 
prod-none(-1.17) 
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Stepwise  Fit  -  New  Survey  Data  -  Consequences  (InhouseNonCore) 

Response: 

Column  59 


Stepwise  Regression  Control 

Prob  to  Enter  0.250 

Prob  to  Leave  0.250 

Direction: 

Rules: 


Current  Estimates 

SSE  DFE 

68.748275  36 


MSE  RSquare 

1.9096743  0.4379 


RSquare  Adj  Cp  AIC 

0.3130  -9.489059  37.07051 


Lock 

X 


X  Intercept 

whattypesystemsavionics{0-1 } 
whattypesystemsembedded{0-1 } 

X  whattypesystemscommunications{0-1} 

whattypesystemsdevice{0-1} 
whattypeshrinkbusiness{0-1} 
whattypeshrinkutilities{0-1 } 
whattypeshrinkinternet{0-1 } 
whattypecomponentdomain{0-1} 
whattypecomponentCASE{0-1} 
whattypecomponentclass{0- 1 } 

X  whattypecomponentOS{0-1} 

X  whattypecomponentdevelopment{0- 1 } 

whattypeenterpriseacctng{0-1 } 
whattypeenterprisemanufact{0-1} 
whattypeenterprisepayroll{0-1 } 

X  whattypeenterprisescripting{0-1 } 

X  whatprocessrequirements{0-1} 

X  whatprocessdesign{0-1} 

X  whatprocesstesting{0-1} 

whatprocessmaintenance{0-1} 

X  whatprocessreengineering{0-1} 

whatprocessappsuppt{0-1 } 
whatprocesstraining{0-1} 
whatprocessspecification{0-1 } 
whatprocesscoding{0- 1 } 
whatprocessfielding{0-1} 
whatprocessCM{0-1} 
whatprocesstoolsuppt{0-1} 
whatprocessSWEngSuppt{0-1} 
whatprocessnone{0-1 } 
whatproductscustom{0-1 } 
whatproductsCOTS{0-1} 
whatproductscommoncust{0-1} 
whatproductsnone{0-1} 


Step  History 

Step  Parameter 

1  whattypecomponentdevelopment{0- 1 } 

2  whattypeenterprisescripting{0-1} 

3  whatprocessreengineering{0-1} 

4  whatprocessdesign{0-1} 

5  whatprocessrequirements{0-1} 

6  whattypecomponentOS{0-1} 

7  whattypesystemscommunications{0-1 } 

8  whatprocesstesting{0-1} 


Estimate  nDF 

3.67722738  1 

0  1 

0  1 

0.45975878  1 

0  1 

0  1 

0  1 

0  1 

0  1 

0  1 

0  1 

-0.7376424  1 

-0.9305903  1 

0  1 

0  1 

0  1 

0.95375526  1 

0.49754348  1 

-1 .0324573  1 

0.44558849  1 

0  1 

-0.6845377  1 

0  1 

0  1 

0  1 

0  1 

0  1 

0  1 

0  1 

0  1 

0  1 

0  1 

0  1 

0  1 

0  1 


SS 

0 

0.21318 

2.378225 

6.309406 

0.000025 

1.800434 

0.273127 

2.56645 
2.168816 
0.103161 
0.938755 
5.829117 
17.64985 
0.012153 
1 .507536 
0.302058 
5.802359 
6.086739 
17.46322 
3.612591 
0.079447 
12.31559 
1.654793 
0.090482 
0.606468 
0.503721 
0.064179 
0.002951 
0.304139 
0.378606 
0.984983 
0.013459 

1 .08586 
0.708652 
2.123857 


"F  Ratio"  "Prob>F" 

0.000  1.0000 

0.109  0.7434 

1.254  0.2704 

3.304  0.0774 

0.000  0.9972 

0.941  0.3386 

0.140  0.7109 

1.357  0.2519 

1.140  0.2929 

0.053  0.8199 

0.485  0.4910 

3.052  0.0891 

9.242  0.0044 

0.006  0.9377 

0.785  0.3818 

0.154  0.6967 

3.038  0.0899 

3.187  0.0826 

9.145  0.0046 

1.892  0.1775 

0.040  0.8417 

6.449  0.0156 

0.863  0.3592 

0.046  0.8312 

0.312  0.5803 

0.258  0.6145 

0.033  0.8575 

0.002  0.9693 

0.156  0.6957 

0.194  0.6625 

0.509  0.4804 

0.007  0.9345 

0.562  0.4586 

0.365  0.5499 

1.116  0.2981 


Action 

Entered 

Entered 

Entered 

Entered 

Entered 

Entered 

Entered 

Entered 


"Sig  Prob"  Seq  SS 

0.0341  12.25472 

0.0872  7.489656 

0.0798  7.482582 

0.1528  4.793857 

0.0817  6.833933 

0.1170  5.292407 

0.0933  5.80309 

0.1775  3.612591 


RSquare  Cp  p 

0.1002  -12.97  2 

0.1614  -12.88  3 

0.2226  -12.78  4 

0.2618  -12  5 

0.3177  -11.74  6 

0.3609  -1 1 .09  7 

0.4084  -10.57  8 

0.4379  -9.489  9 


Appendix  B 


Response  Column  59 
Actual  by  Predicted  Plot 

8 
7 

1  6 

<5 

05 

in  4 

c 

I3 

<3  2 

i 

0 

1  2  3  4  5  6  7 

Column  59  Predicted  P=0.0043  RSq=0.44 
RMSE=1 .3819 


Summary  of  Fit 


RSquare  0.437923 

RSquare  Adj  0.313017 

Root  Mean  Square  Error  1.38191 

Mean  of  Response  3.355556 

Observations  (or  Sum  Wgts)  45 


Analysis  of  Variance 

Source 

DF 

Sum  of  Squares 

Mean  Square 

Model 

8 

53.56284 

6.69535 

Error 

36 

68.74827 

1 .90967 

C.  Total 

44 

122.31111 

F  Ratio 
3.5060 
Prob  >  F 
0.0043 


Lack  Of  Fit 

Source 
Lack  Of  Fit 
Pure  Error 
Total  Error 


DF 

15 

21 

36 


Sum  of  Squares  Mean  Square 

35.057798  2.33719 

33.690476  1.60431 

68.748275 


F  Ratio 
1.4568 
Prob  >  F 
0.2094 
Max  RSq 
0.7246 


Parameter  Estimates 


Term 

Estimate 

Std  Error 

t  Ratio  Prob>|t| 

Intercept 

2.6486457 

0.372972 

7.10  <.0001 

whattypesystemscommunications[1-0] 

-0.919518 

0.505878 

-1.82  0.0774 

whattypecomponentOS[1  -0] 

1.4752848 

0.84441 1 

1.75  0.0891 

whattypecomponentdevelopment[1-0] 

1.8611806 

0.612206 

3.04  0.0044 

whattypeenterprisescripting[1  -0] 

-1.907511 

1 .09432 

-1.74  0.0899 

whatprocessrequirements[1-0] 

-0.995087 

0.557376 

-1.79  0.0826 

whatprocessdesign[1  -0] 

2.0649145 

0.682841 

3.02  0.0046 

whatprocesstesting[1  -0] 

-0.891177 

0.647939 

-1.38  0.1775 

whatprocessreengineering[1  -0] 

1.3690755 

0.539113 

2.54  0.0156 

Effect  T ests 

Source 

Nparm  DF 

Sum  of  Squares 

F  Ratio  Prob  >  F 

whattypesystemscommunications 

1  1 

6.309406 

3.3039 

0.0774 

whattypecomponentOS 

1  1 

5.829117 

3.0524 

0.0891 

whattypecomponentdevelopment 

1  1 

17.649854 

9.2423 

0.0044 

whattypeenterprisescripting 

1  1 

5.802359 

3.0384 

0.0899 

whatprocessrequirements 

1  1 

6.086739 

3.1873 

0.0826 

whatprocessdesign 

1  1 

17.463218 

9.1446 

0.0046 

whatprocesstesting 

1  1 

3.612591 

1.8917 

0.1775 

whatprocessreengineering 

1  1 

12.315587 

6.4491 

0.0156 

InhouseNonCore  =  2.65  +  sys-comm(-0.92)  +  comp-OS(1 .48) 

+  comp-dev(1.86) 

+  ent-script(-1.91) 

+  proc- 

proc-des(2.06)  +  proc-test(-0.89)  +  proc-reeng(1.37) 


Page  7 


Stepwise  Fit  -  New  Survey  Data  -  Consequences  (InhouseTurnover) 

Response: 

Column  59 

Stepwise  Regression  Control 

Prob  to  Enter  0.250 

Prob  to  Leave  0.250 

Direction: 

Rules: 


Current  Estimates 

SSE  DFE 

57.912676  34 


MSE  RSquare  RSquare  Adj 

1.703314  0.5111  0.3961 


Cp  AIC 

-10.1893  30.80266 


Lock 

X 


X  Intercept 

whattypesystemsavionics{0-1 } 
whattypesystemsembedded{0-1 } 
whattypesystemscommunications{0-1} 
whattypesystemsdevice{0-1} 
whattypeshrinkbusiness{0-1} 
whattypeshrinkutilities{0-1 } 
whattypeshrinkinternet{0-1 } 
whattypecomponentdomain{0-1} 
whattypecomponentCASE{0-1} 
whattypecomponentclass{0- 1 } 

X  whattypecomponentOS{0-1} 

X  whattypecomponentdevelopment{0- 1 } 

whattypeenterpriseacctng{0-1 } 
whattypeenterprisemanufact{0-1} 
whattypeenterprisepayroll{0-1 } 

X  whattypeenterprisescripting{0-1 } 

X  whatprocessrequirements{0-1} 

X  whatprocessdesign{0-1} 

X  whatprocesstesting{0-1} 

whatprocessmaintenance{0-1} 

X  whatprocessreengineering{0-1} 

whatprocessappsuppt{0-1 } 
whatprocesstraining{0-1} 

X  whatprocessspecification{0-1} 

whatprocesscoding{0- 1 } 
whatprocessfielding{0-1} 
whatprocessCM{0-1} 
whatprocesstoolsuppt{0-1} 
whatprocessSWEngSuppt{0-1} 
whatprocessnone{0-1 } 
whatproductscustom{0-1 } 
whatproductsCOTS{0-1} 
whatproductscommoncust{0-1} 
whatproductsnone{0-1} 


Step  History 

Step  Parameter 

1  whattypecomponentdevelopment{0- 1 } 

2  whattypeenterprisescripting{0-1} 

3  whatprocessdesign{0-1} 

4  whatprocessreengineering{0-1} 

5  whatprocessrequirements{0-1} 

6  whattypecomponentOS{0-1} 

7  whatprocesstesting{0-1} 

8  whatprocessspecification{0-1} 


Action 

Entered 

Entered 

Entered 

Entered 

Entered 

Entered 

Entered 

Entered 


Estimate  nDF 

4.13724751  1 

0  1 

0  1 

0  1 

0  1 

0  1 

0  1 

0  1 

0  1 

0  1 

0  1 

-0.8695159  1 

-1.1051889  1 

0  1 

0  1 

0  1 

0.82642342  1 

0.52842467  1 

-1 .2358544  1 

0.44130819  1 

0  1 

-0.5557235  1 

0  1 

0  1 

0.36214878  1 

0  1 

0  1 

0  1 

0  1 

0  1 

0  1 

0  1 

0  1 

0  1 

0  1 


SS 

0 

0.000166 
1.543722 
1.454364 
0.459877 
0.394162 
0.05437 
0.405376 
0.000589 
0.019528 
0.869941 
7.386944 
22.17177 
0.193319 
1.009041 
1.184276 
4.283826 
6.477252 
20.95883 
3.48425 
0.21965 
8.417279 
1.015946 
0.020602 
3.103879 
0.839887 
0.001083 
0.045667 
0.715542 
0.305319 
0.662922 
0.014433 
2.25671 1 
0.012959 
0.724134 


"F  Ratio" 
0.000 
0.000 
0.904 
0.850 
0.264 
0.226 
0.031 
0.233 
0.000 
0.011 
0.503 

4.337 
13.017 

0.111 

0.585 

0.689 

2.515 

3.803 

12.305 

2.046 

0.126 

4.942 

0.589 

0.012 

1.822 

0.486 

0.001 

0.026 

0.413 

0.175 

0.382 

0.008 

1.338 
0.007 
0.418 


'Prob>F" 

1.0000 

0.9923 

0.3487 

0.3632 

0.6107 

0.6375 

0.8613 

0.6328 

0.9855 

0.9166 

0.4830 

0.0449 

0.0010 

0.7416 

0.4497 

0.4125 

0.1220 

0.0595 

0.0013 

0.1618 

0.7253 

0.0330 

0.4482 

0.9144 

0.1860 

0.4908 

0.9803 

0.8728 

0.5250 

0.6785 

0.5407 

0.9283 

0.2557 

0.9320 

0.5225 


'Sig  Prob"  Seq  SS 

0.0111  17.43809 

0.0471  9.590618 

0.0925  6.479512 

0.0937  6.132822 

0.0302  9.513617 

0.0857  5.532208 

0.2165  2.761694 

0.1860  3.103879 


RSquare  Cp  p 

0.1472  -13.16  2 

0.2282  -13.62  3 

0.2829  -13.27  4 

0.3346  -12.84  5 

0.4149  -13.27  6 

0.4616  -12.69  7 

0.4849  -11.4  8 

0.5111  -10.19  9 


Appendix  B 


Response  Column  59 
Actual  by  Predicted  Plot 

8 
7 

1  6 

<5 

05 

in  4 

c 

I3 

<3  2 

i 

0 

1  2  3  4  5  6  7 

Column  59  Predicted  P=0.0009  RSq=0.51 
RMSE=1 .3051 


Summary  of  Fit 


RSquare  0.511142 

RSquare  Adj  0.3961 1 6 

Root  Mean  Square  Error  1 .3051 1 1 

Mean  of  Response  3.418605 

Observations  (or  Sum  Wgts)  43 


Analysis  of  Variance 

Source 

DF 

Sum  of  Squares 

Mean  Square 

Model 

8 

60.55244 

7.56906 

Error 

34 

57.91268 

1 .70331 

C.  Total 

42 

118.46512 

F  Ratio 
4.4437 
Prob  >  F 
0.0009 


Lack  Of  Fit 

Source 
Lack  Of  Fit 
Pure  Error 
Total  Error 


DF 

13 

21 

34 


Sum  of  Squares 
27.571009 
30.341667 
57.912676 


Mean  Square 
2.12085 
1.44484 


F  Ratio 
1.4679 
Prob  >  F 
0.2101 
Max  RSq 
0.7439 


Parameter  Estimates 

Term 

Intercept 

whattypecomponentOS[1  -0] 
whattypecomponentdevelopment[1-0] 
whattypeenterprisescripting[1  -0] 
whatprocessrequirements[1-0] 
whatprocessdesign[1  -0] 
whatprocesstesting[1  -0] 
whatprocessreengineering[1  -0] 
whatprocessspecification[1  -0] 


Estimate 
2.5292699 
1.7390318 
2.2103778 
-1 .652847 
-1 .056849 
2.4717088 
-0.882616 
1.1114471 
-0.724298 


Std  Error  t  Ratio  Prob>|t| 
0.363179  6.96  <.0001 

0.835069  2.08  0.0449 

0.612652  3.61  0.0010 

1.042231  -1.59  0.1220 

0.541957  -1.95  0.0595 

0.70463  3.51  0.0013 

0.617113  -1.43  0.1618 

0.499977  2.22  0.0330 

0.536552  -1.35  0.1860 


Effect  T ests 

Source  Nparm  DF 

whattypecomponentOS  1  1 

whattypecomponentdevelopment  1  1 

whattypeenterprisescripting  1  1 

whatprocessrequirements  1  1 

whatprocessdesign  1  1 

whatprocesstesting  1  1 

whatprocessreengineering  1  1 

whatprocessspecification  1  1 


InhouseTurnover  =  2.53  +  comp-OS(1.74)  +  comp-dev(2.21) 
test(-0.88)  +  proc-reeng(l.ll)  +  proc-spec(-0.72) 


Sum  of  Squares  F  Ratio 

7.386944  4.3368 

22.171768  13.0168 

4.283826  2.5150 

6.477252  3.8027 

20.958829  12.3047 

3.484250  2.0456 

8.417279  4.9417 

3.103879  1.8223 


ent-script(-1.65)  +  proc-req(- 


Prob  >  F 
0.0449 
0.0010 
0.1220 
0.0595 
0.0013 
0.1618 
0.0330 
0.1860 

.06)  +  proc-des(2.47)  +  proc- 
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Stepwise  Fit  -  New  Survey  Data  -  Consequences  (LearningCurve) 

Response: 

Column  59 


Stepwise  Regression  Control 

Prob  to  Enter  0.250 

Prob  to  Leave  0.250 

Direction: 

Rules: 

Current  Estimates 


SSE  DFE  MSE  RSquare 

RSquare  Adj 

Cp 

AIC 

41.438313  34  1.2187739 

0.5889 

0.4680 

-7.750854 

18.28945 

Lock 

Entered  Parameter 

Estimate 

nDF 

SS 

"F  Ratio" 

"Prob>F" 

X 

X  Intercept 

1.23539973 

1 

0 

0.000 

1.0000 

whattypesystemsavionics{0-1 } 

0 

1 

0.000299 

0.000 

0.9878 

whattypesystemsembedded{0-1 } 

0 

1 

0.259381 

0.208 

0.6514 

whattypesystemscommunications{0-1} 

0 

1 

1.366663 

1.125 

0.2964 

whattypesystemsdevice{0- 1 } 

0 

1 

0.452377 

0.364 

0.5503 

whattypeshrinkbusiness{0-1} 

0 

1 

0.016396 

0.013 

0.9097 

X  whattypeshrinkutilities{0-1} 

-0.7719827 

1 

6.343735 

5.205 

0.0289 

whattypeshrinkinternet{0-1 } 

0 

1 

0.579157 

0.468 

0.4988 

whattypecomponentdomain{0-1} 

0 

1 

1.362002 

1.122 

0.2973 

X  whattypecomponentCASE{0-1} 

1.86132029 

1 

21.98134 

18.036 

0.0002 

X  whattypecomponentclass{0-1} 

0.94998532 

1 

7.770041 

6.375 

0.0164 

whattypecomponentOS{0-1 } 

0 

1 

0.732961 

0.594 

0.4463 

whattypecomponentdevelopment{0-1} 

0 

1 

0.004251 

0.003 

0.9540 

whattypeenterpriseacctng{0-1 } 

0 

1 

0.428255 

0.345 

0.5612 

whattypeenterprisemanufact{0-1 } 

0 

1 

0.086193 

0.069 

0.7947 

X  whattypeenterprisepayroll{0-1} 

0.56870351 

1 

3.55341 

2.916 

0.0968 

whattypeenterprisescripting{0-1} 

0 

1 

0.026351 

0.021 

0.8857 

whatprocessrequirements{0-1} 

0 

1 

0.180064 

0.144 

0.7067 

whatprocessdesign{0- 1 } 

0 

1 

0.727565 

0.590 

0.4480 

whatprocesstesting{0- 1 } 

0 

1 

0.690856 

0.560 

0.4598 

whatprocessmaintenance{0-1} 

0 

1 

0.151973 

0.121 

0.7297 

whatprocessreengineering{0-1 } 

0 

1 

0.535668 

0.432 

0.5155 

X  whatprocessappsuppt{0-1} 

-0.3219125 

1 

3.020342 

2.478 

0.1247 

X  whatprocesstraining{0-1} 

0.34924856 

1 

3.485787 

2.860 

0.1000 

whatprocessspecification{0-1 } 

0 

1 

0.559616 

0.452 

0.5062 

X  whatprocesscoding{0-1} 

-0.6474073 

1 

8.693202 

7.133 

0.0115 

whatprocessfielding{0-1} 

0 

1 

0.654328 

0.529 

0.4720 

X  whatprocessCM{0-1} 

0.30777557 

1 

2.27584 

1.867 

0.1808 

whatprocesstoolsuppt{0-1} 

0 

1 

0.303821 

0.244 

0.6248 

whatprocessSWEngSuppt{0- 1 } 

0 

1 

1.156005 

0.947 

0.3376 

X  whatprocessnone{0-1} 

0.57478303 

1 

1.953073 

1.602 

0.2142 

whatproductscustom{0-1 } 

0 

1 

1.291256 

1.061 

0.3104 

whatproductsCOTS{0-1} 

0 

1 

1.045496 

0.854 

0.3621 

X  whatproductscommoncust{0-1} 

-0.9563475 

1 

17.95584 

14.733 

0.0005 

whatproductsnone{0-1} 

0 

1 

0.734908 

0.596 

0.4457 

Step  History 

Step 

Parameter 

Action 

"Sig  Prob" 

Seq  SS 

RSquare 

Cp 

P 

1 

whatproductsnone{0-1} 

Entered 

0.0351 

10 

0.0992 

-7.586 

2 

2 

whattypecomponentOS{0-1 } 

Entered 

0.0268 

10.12252 

0.1996 

-9.311 

3 

3 

whattypecomponentclass{0-1 } 

Entered 

0.1058 

5.046105 

0.2497 

-9.168 

4 

4 

whattypecomponentCASE{0-1} 

Entered 

0.1029 

4.924839 

0.2985 

-8.98 

5 

5 

whatproductscommoncust{0-1} 

Entered 

0.0378 

7.49027 

0.3729 

-9.737 

6 

6 

whatprocesscoding{0-1} 

Entered 

0.0351 

7.060514 

0.4429 

-10.33 

7 

7 

whatproductsnone{0-1 } 

Removed 

0.4083 

1.032976 

0.4327 

-11.95 

6 

8 

whattypeshrinkutilities{0-1} 

Entered 

0.0305 

6.711225 

0.4992 

-12.42 

7 

9 

whatprocessnone{0-1} 

Entered 

0.2038 

2.184358 

0.5209 

-11.23 

8 

10 

whattypeenterprisepayroll{0-1 } 

Entered 

0.2406 

1.83699 

0.5391 

-9.904 

9 

11 

whatprocesstraining{0-1 } 

Entered 

0.2133 

2.039165 

0.5594 

-8.655 

10 

12 

whattypecomponentOS{0-1 } 

Removed 

0.3283 

1.24708 

0.5470 

-10.2 

9 

13 

whatprocessappsuppt{0-1 } 

Entered 

0.2198 

1.949916 

0.5663 

-8.913 

10 

14 

whatprocessC  M{0- 1 } 

Entered 

0.1808 

2.27584 

0.5889 

-7.751 

11 
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Response  Column  59 
Actual  by  Predicted  Plot 


la 

O 


1  2  3  4  5  6  7 

Column  59  Predicted  P=0.0002  RSq=0.59 
RMSE=1.104 


Summary  of  Fit 


RSquare  0.588906 

RSquare  Adj  0.467996 

Root  Mean  Square  Error  1.103981 

Mean  of  Response  3.933333 

Observations  (or  Sum  Wgts)  45 


Analysis  of  Variance 

Source  DF  Sum  of  Squares 

Model  10  59.36169 

Error  34  41.43831 

C.  Total  44  100.80000 


Mean  Square  F  Ratio 

5.93617  4.8706 

1.21877  Prob  >  F 

0.0002 


Lack  Of  Fit 


Source  DF 

Lack  Of  Fit  16 

Pure  Error  18 

Total  Error  34 


Parameter  Estimates 

Term 

Intercept 

whattypeshrinkutilities[1  -0] 
whattypecomponentCASE[1  -0] 
whattypecomponentclass[1  -0] 
whattypeenterprisepayroll[1  -0] 
whatprocessappsuppt[1-0] 
whatprocesstraining[1-0] 
whatprocesscoding[1  -0] 
whatprocessCM[1  -0] 
whatprocessnone[1  -0] 
whatproductscommoncust[1-0] 
Effect  Tests 
Source 

whattypeshrinkutilities 

whattypecomponentCASE 

whattypecomponentclass 

whattypeenterprisepayroll 

whatprocessappsuppt 

whatprocesstraining 

whatprocesscoding 

whatprocessCM 

whatprocessnone 

whatproductscommoncust 


F  Ratio 
0.9418 
Prob  >  F 
0.5445 
Max  RSq 
0.7762 


Estimate  Std  Error  t  Ratio  Prob>|t| 
3.1495661  0.46397  6.79  <.0001 

1.5439654  0.676747  2.28  0.0289 

-3.722641  0.876568  -4.25  0.0002 

-1.899971  0.752483  -2.52  0.0164 

-1.137407  0.666124  -1.71  0.0968 

0.6438249  0.408979  1.57  0.1247 

-0.698497  0.413024  -1.69  0.1000 

1.2948145  0.484819  2.67  0.0115 

-0.615551  0.450459  -1.37  0.1808 

-1.149566  0.908105  -1.27  0.2142 

1.9126951  0.498316  3.84  0.0005 


Sum  of  Squares 
18.882758 
22.555556 
41.438313 


Mean  Square 
1.18017 
1.25309 


Nparm  DF 

1  1 

1  1 

1  1 

1  1 

1  1 

1  1 

1  1 

1  1 

1  1 

1  1 


Sum  of  Squares 
6.343735 
21.981339 
7.770041 
3.553410 
3.020342 
3.485787 
8.693202 
2.275840 
1.953073 
17.955837 


F  Ratio  Prob  >  F 

5.2050  0.0289 

18.0356  0.0002 

6.3753  0.0164 

2.9156  0.0968 

2.4782  0.1247 

2.8601  0.1000 

7.1327  0.0115 

1.8673  0.1808 

1.6025  0.2142 

14.7327  0.0005 


LearningCurve  =  3.15  +  shrink-util(1 .54)  +  comp-CASE(-3.72)  +  comp-class(-1 .90)  +  ent-pay(-1.14)  +  proc-appsup(0.64)  + 
proc-train(-0.70)  +  proc-coding(1.29)  +  proc-CM(-0.62)  +  proc-none(-1.15)  +  prod-comcust(1.91) 
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Stepwise  Fit  -  New  Survey  Data  -  Consequences  (Risk) 

Response: 

Column  59 


Stepwise  Regression  Control 

Prob  to  Enter  0.250 

Prob  to  Leave  0.250 

Direction: 

Rules: 


Current  Estimates 


SSE 

DFE  MSE  RSquare 

RSquare  Adj 

Cp 

AIC 

63.741188 

39  1.6343894  0.3799 

0.3005  -17.17227 

27.66758 

Lock 

Entered 

Parameter 

Estimate 

nDF 

SS 

X 

X 

Intercept 

4.26591333 

1 

0 

whattypesystemsavionics{0-1 } 

0 

1 

0.000668 

X 

whattypesystemsembedded{0-1 } 

-0.506609 

1 

2.600154 

whattypesystemscommunications{0-1} 

0 

1 

0.023497 

whattypesystemsdevice{0- 1 } 

0 

1 

0.00048 

whattypeshrinkbusiness{0-1} 

0 

1 

0.010745 

X 

whattypeshrinkutilities{0-1} 

-0.756609 

1 

5.799576 

whattypeshrinkinternet{0-1 } 

0 

1 

0.400015 

whattypecomponentdomain{0-1} 

0 

1 

0.112491 

whattypecomponentCASE{0-1} 

0 

1 

1.198648 

whattypecomponentclass{0- 1 } 

0 

1 

0.952848 

X 

whattypecomponentOS{0-1 } 

0.74284678 

1 

5.689865 

whattypecomponentdevelopment{0-1} 

0 

1 

0.205341 

whattypeenterpriseacctng{0-1 } 

0 

1 

1 .843708 

whattypeenterprisemanufact{0-1} 

0 

1 

0.169408 

X 

whattypeenterprisepayroll{0-1} 

0.49502384 

1 

3.338614 

whattypeenterprisescripting{0-1} 

0 

1 

2.07651 

whatprocessrequirements{0-1} 

0 

1 

0.04932 

whatprocessdesign{0- 1 } 

0 

1 

0.064219 

whatprocesstesting{0- 1 } 

0 

1 

0.04498 

whatprocessmaintenance{0-1} 

0 

1 

0.008015 

whatprocessreengineering{0-1} 

0 

1 

1.359713 

whatprocessappsuppt{0-1 } 

0 

1 

0.383683 

whatprocesstraining{0-1} 

0 

1 

0.01775 

whatprocessspecification{0-1 } 

0 

1 

0.013787 

whatprocesscoding{0- 1 } 

0 

1 

0.1595 

whatprocessfielding{0-1) 

0 

1 

0.086778 

whatprocessCM{0-1} 

0 

1 

0.335666 

whatprocesstoolsuppt{0-1} 

0 

1 

0.024257 

whatprocessSWEngSuppt{0- 1 } 

0 

1 

0.282038 

whatprocessnone{0-1 } 

0 

1 

2.07651 

whatproductscustom{0-1 } 

0 

1 

0.491787 

whatproductsCOTS{0-1} 

0 

1 

1.618142 

X 

whatproductscommoncust{0-1} 

-0.7581899 

1 

14.38397 

whatproductsnone{0-1} 

0 

1 

1.457479 

Step  History 

Step 

Parameter 

Action 

"Sig  Prob" 

Seq  SS 

RSquare 

1 

whattypecomponentOS{0-1 } 

Entered 

0.0136 

13.72857 

0.1335 

2 

whatproductscommoncust{0-1} 

Entered 

0.0281 

9.765056 

0.2285 

3 

whattypeshrinkutilities{0-1 } 

Entered 

0.0241 

9.356104 

0.3195 

4 

whattypeenterprisepayroll{0-1 } 

Entered 

0.1480 

3.608927 

0.3547 

5 

whattypesystemsembedded{0-1 } 

Entered 

0.2147 

2.600154 

0.3799 

"F  Ratio"  "Prob>F" 

0.000  1.0000 

0.000  0.9842 

1.591  0.2147 

0.014  0.9064 

0.000  0.9866 

0.006  0.9366 

3.548  0.0671 

0.240  0.6270 

0.067  0.7969 

0.728  0.3988 

0.577  0.4523 

3.481  0.0696 

0.123  0.7279 

1.132  0.2941 

0.101  0.7521 

2.043  0.1609 

1.280  0.2651 

0.029  0.8647 

0.038  0.8458 

0.027  0.8707 

0.005  0.9452 

0.828  0.3685 

0.230  0.6342 

0.011  0.9186 

0.008  0.9282 

0.095  0.7592 

0.052  0.8212 

0.201  0.6563 

0.014  0.9049 

0.169  0.6834 

1.280  0.2651 

0.295  0.5899 

0.990  0.3261 

8.801  0.0051 

0.889  0.3516 


Cp  p 

-18.88  2 

-19.31  3 

-19.63  4 

-18.53  5 

-17.17  6 
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Response  Column  59 
Actual  by  Predicted  Plot 

8 
7 

1  6 

<5 

05 

in  4 

c 

I3 

° 


i 

0 

1  2  3  4  5  6  7 

Column  59  Predicted  P=0.0017  RSq=0.38 
RMSE=1 .2784 


Summary  of  Fit 

RSquare  0.37995 

RSquare  Adj  0.300456 

Root  Mean  Square  Error  1 .278432 

Mean  of  Response  3.733333 

Observations  (or  Sum  Wgts)  45 


Analysis  of  Variance 


Source  DF 

Sum  of  Squares 

Mean  Square 

F  Ratio 

Model  5 

39.05881 

7.81176 

4.7796 

Error  39 

63.74119 

1 .63439 

Prob  >  F 

C.  Total  44 

102.80000 

0.0017 

Lack  Of  Fit 

Source  DF 

Sum  of  Squares 

Mean  Square  F  Ratio 

Lack  Of  Fit  3 

8.794098 

2.93137  1.9206 

Pure  Error  36 

54.947090 

1.52631  Prob  >  F 

Total  Error  39 

63.741188 

0.1437 
Max  RSq 

0.4655 

Parameter  Estimates 

Term 

Estimate 

Std  Error  t  Ratio  Prob>|t| 

Intercept 

3.4823761 

0.240018 

14.51  <.0001 

whattypesystemsembedded[1  -0] 

1.0132179 

0.803306 

1.26  0.2147 

whattypeshrinkutilities[1  -0] 

1.5132179 

0.803306 

1.88  0.0671 

whattypecomponentOS[1  -0] 

-1.485694 

0.796262 

-1.87  0.0696 

whattypeenterprisepayroll[1  -0] 

-0.990048 

0.692709 

-1.43  0.1609 

whatproductscommoncust[1-0] 

1.5163798 

0.511148 

2.97  0.0051 

Effect  T ests 

Source 

Nparm 

DF 

Sum  of  Squares 

F  Ratio 

Prob  >  F 

whattypesystemsembedded 

1 

1 

2.600154 

1.5909 

0.2147 

whattypeshrinkutilities 

1 

1 

5.799576 

3.5485 

0.0671 

whattypecomponentOS 

1 

1 

5.689865 

3.4813 

0.0696 

whattypeenterprisepayroll 

1 

1 

3.338614 

2.0427 

0.1609 

whatproductscommoncust 

1 

1 

14.383974 

8.8008 

0.0051 

Risk  =  3.48  +  sys-embed(I.OI)  +  shrink-util(1.51)  +  comp-OS(-1 .49)  +  ent-pay(-0.99)  +  prod-comcust(1.52) 
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Stepwise  Fit  -  New  Survey  Data  -  Consequences  (Quality) 

Response: 

Column  40 


Stepwise  Regression  Control 

Prob  to  Enter  0.250 

Prob  to  Leave  0.250 

Direction: 

Rules: 


Current  Estimates 

SSE  DFE 

34.726132  34 

Lock  Entered  Parameter 

X  X 


MSE 

1.0213568 


RSquare 

0.4774 


Intercept 

whattypesystemsavionics{0-1 } 
whattypesystemsembedded{0-1 } 
whattypesystemscommunications{0-1} 
whattypesystemsdevice{0- 1 } 
whattypeshrinkbusiness{0-1} 
whattypeshrinkutilities{0-1} 
whattypeshrinkinternet{0-1} 
whattypecomponentdomain{0-1} 
whattypecomponentCASE{0-1 } 
whattypecomponentclass{0- 1 } 
whattypecomponentOS{0-1 } 
whattypecomponentdevelopment{0-1} 
whattypeenterpriseacctng{0-1 } 
whattypeenterprisemanufact{0-1} 
whattypeenterprisepayroll{0-1 } 
whattypeenterprisescripting{0-1} 
whatprocessrequirements{0-1 } 
whatprocessdesign{0- 1 } 
whatprocesstesting{0- 1 } 
whatprocessmaintenance{0-1} 
whatprocessreengineering{0-1} 
whatprocessappsuppt{0-1 } 
whatprocesstraining{0-1} 
whatprocessspecification{0-1 } 
whatprocesscoding{0- 1 } 
whatprocessfielding{0-1} 
whatprocessCM{0-1 } 
whatprocesstoolsuppt{0-1} 
whatprocessSWEngSuppt{0- 1 } 
whatprocessnone{0-1 } 
whatproductscustom{0-1 } 
whatproductsCOTS{0-1} 
whatproductscommoncust{0-1} 
whatproductsnone{0-1} 


RSquare  Adj  Cp 

0.3236  -3.414153 

Estimate  nDF 
6.72871893  1 

0  1 

0.63501439  1 

0  1 

-0.9145133  1 

0  1 

-0.4471425  1 

-0.4337958  1 

0  1 

-0.7193332  1 

0.43836462  1 


Step  History 

Step  Parameter 

1  whattypesystemsembedded{0-1} 

2  whattypecomponentOS{0-1} 

3  whatproductsnone{0-1} 

4  whattypeshrinkutilities{0-1} 

5  whatproductscustom{0-1} 

6  whattypesystemsdevice{0-1} 

7  whattypeshrinkinternet{0-1} 

8  whatprocesscoding{0-1} 

9  whatprocessspecification{0-1} 

10  whattypecomponentOS{0-1} 

11  whattypecomponentCASE{0-1} 

12  whattypecomponentclass{0-1} 


Action 

Entered 

Entered 

Entered 

Entered 

Entered 

Entered 

Entered 

Entered 

Entered 

Removed 

Entered 

Entered 


-0.4426799 

0.66742175 

0 

0 

0 

0 

0 

-0.5496736 

0 

0 

-0.5332325 

"Sig  Prob" 
0.0742 
0.1257 
0.1555 
0.1411 
0.1315 
0.2250 
0.1645 
0.1273 
0.0894 
0.5261 
0.1153 
0.1861 


Seq  SS 
4.801587 
3.386447 
2.83092 
2.956259 
3.009167 
1.90384 
2.452342 
2.859126 
3.38787 
0.455376 
2.726166 
1.859963 


AIC 

10.33735 

SS 

0 

0.091289 
3.517759 
0.37393 
10.22455 
0.317767 
1.852111 
4.585903 
0.113248 
3.05832 
1 .859963 
0.122281 
0.277549 
0.586581 
0.230344 
0.105161 
0.912346 
0.471358 
0.647233 
0.166293 
0.021091 
0.05039 
0.153121 
0.430868 
6.209551 
10.1736 
0.445555 
0.01652 
0.199607 
0.239718 
0.000075 
6.77432 
0.895292 
0.415275 
3.600719 

RSquare 
0.0723 
0.1232 
0.1658 
0.2103 
0.2556 
0.2843 
0.3212 
0.3642 
0.4152 
0.4083 
0.4494 
0.4774 


"F  Ratio" 
0.000 
0.087 
3.444 
0.359 
10.011 
0.305 
1.813 
4.490 
0.108 
2.994 
1.821 
0.117 
0.266 
0.567 
0.220 
0.100 
0.890 
0.454 
0.627 
0.159 
0.020 
0.048 
0.146 
0.415 
6.080 
9.961 
0.429 
0.016 
0.191 
0.229 
0.000 
6.633 
0.873 
0.399 
3.525 


Cp 

-6.233 

-6.143 

-5.74 

-5.407 

-5.104 

-4.178 

-3.561 

-3.174 

-3.084 

-4.828 

-4.365 

-3.414 


"Prob>F" 

1.0000 

0.7699 

0.0722 

0.5530 

0.0033 

0.5846 

0.1870 

0.0415 

0.7445 

0.0926 

0.1861 

0.7349 

0.6095 

0.4568 

0.6419 

0.7535 

0.3522 

0.5051 

0.4342 

0.6928 

0.8882 

0.8280 

0.7047 

0.5241 

0.0189 

0.0033 

0.5171 

0.9010 

0.6651 

0.6351 

0.9933 

0.0145 

0.3568 

0.5317 

0.0690 

P 

2 

3 

4 

5 

6 

7 

8 

9 

10 

9 

10 

11 
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Response  Column  59 
Actual  by  Predicted  Plot 

I  8n - 


- 1 - 1 - 1 - 1 - 

2  3  4  5  6  7 

Column  59  Predicted  P=0.0065  RSq=0.48 
RMSE=1.0106 


Summary  of  Fit 


RSquare  0.477366 

RSquare  Adj  0.32365 

Root  Mean  Square  Error  1.010622 

Mean  of  Response  4.888889 

Observations  (or  Sum  Wgts)  45 


Analysis  of  Variance 


Source  DF 

Model  10 

Error  34 

C.  Total  44 


Sum  of  Squares 
31.718312 
34.726132 
66.444444 


Mean  Square  F  Ratio 

3.17183  3.1055 

1.02136  Prob  >  F 

0.0065 


Lack  Of  Fit 

Source 
Lack  Of  Fit 
Pure  Error 
Total  Error 


DF  Sum  of  Squares 

13  7.035656 

21  27.690476 

34  34.726132 


Mean  Square  F  Ratio 

0.54120  0.4104 

1.31859  Prob  >  F 

0.9491 
Max  RSq 
0.5833 


Parameter  Estimates 

Term 

Intercept 

whattypesystemsembedded[1  -0] 
whattypesystemsdevice[1  -0] 
whattypeshrinkutilities[1  -0] 
whattypeshrinkinternet[1  -0] 
whattypecomponentCASE[1  -0] 
whattypecomponentclass[1  -0] 
whatprocessspecification[1  -0] 
whatprocesscoding[1  -0] 
whatproductscustom[1-0] 
whatproductsnone[1-0] 


Estimate 
4.4291488 
-1 .270029 
1.8290266 
0.894285 
0.8675917 
1.4386663 
-0.876729 
0.8853598 
-1 .334844 
1 .0993473 
1.066465 


Std  Error 
0.403271 
0.684336 
0.578078 
0.664097 
0.409441 
0.831395 
0.649684 
0.359069 
0.422943 
0.426865 
0.56799 


t  Ratio  Prob>|t| 

10.98  <.0001 

-1.86  0.0722 

3.16  0.0033 

1.35  0.1870 

2.12  0.0415 

1.73  0.0926 

-1.35  0.1861 

2.47  0.0189 

-3.16  0.0033 

2.58  0.0145 

1.88  0.0690 


Effect  Tests 

Source 

Whattypesystemsembedded 

Whattypesystemsdevice 

Whattypeshrinkutilities 

Whattypeshrinkintemet 

whattypecomponentCASE 

Whattypecomponentclass 

Whatprocessspecification 

Whatprocesscod  i  ng 

Whatproductscustom 

Whatproductsnone 


Nparm  DF 

1  1 

1  1 

1  1 

1  1 

1  1 

1  1 

1  1 

1  1 

1  1 

1  1 


Sum  of  Squares 
3.517759 
10.224550 
1.852111 
4.585903 
3.058320 
1 .859963 
6.209551 
10.173597 
6.774320 
3.600719 


F  Ratio  Prob  >  F 

3.4442  0.0722 

10.0108  0.0033 

1.8134  0.1870 

4.4900  0.0415 

2.9944  0.0926 

1.8211  0.1861 

6.0797  0.0189 

9.9609  0.0033 

6.6327  0.0145 

3.5254  0.0690 


Quality  =  4.43  +  sys-embed(-1 .27)  +  sys-dev(1 .83)  +  shrink-util(0.89)  +  shrink-int(0.87)  +  comp-CASE(1.44)  +  comp- 
class(-0.88)  +  proc-spec(0.89)  +  proc-coding(-1.33)  +  prod-cust(I.IO)  +  prod-none(1.07) 
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Stepwise  Fit  -  New  Survey  Data  -  Consequences  (Rework) 

Response: 

Column  59 


Stepwise  Regression  Control 

Prob  to  Enter  0.250 

Prob  to  Leave  0.250 

Direction: 

Rules: 


Current  Estimates 

SSE  DFE 

59.202975  37 


MSE  RSquare 

1.6000804  0.3207 


RSquare  Adj 
0.2106 


Cp  AIC 

-6.613906  27.05842 


Lock 

X 


X  Intercept 

X  whattypesystemsavionics{0-1} 

whattypesystemsembedded{0-1 } 
whattypesystemscommunications{0-1} 
whattypesystemsdevice{0-1} 
whattypeshrinkbusiness{0-1} 

X  whattypeshrinkutilities{0-1} 

whattypeshrinkinternet{0-1 } 
whattypecomponentdomain{0-1} 
whattypecomponentCASE{0-1} 
whattypecomponentclass{0- 1 } 
whattypecomponentOS{0-1 } 

X  whattypecomponentdevelopment{0-1} 

whattypeenterpriseacctng{0-1 } 
whattypeenterprisemanufact{0-1} 
whattypeenterprisepayroll{0-1 } 
whattypeenterprisescripting{0-1} 
whatprocessrequirements{0-1 } 
whatprocessdesign{0- 1 } 
whatprocesstesting{0- 1 } 
whatprocessmaintenance{0-1} 
whatprocessreengineering{0-1} 

X  whatprocessappsuppt{0-1} 

whatprocesstraining{0-1} 
whatprocessspecification{0-1 } 
whatprocesscoding{0- 1 } 
whatprocessfielding{0-1} 
whatprocessCM{0-1} 
whatprocesstoolsuppt{0-1} 

X  whatprocessSWEngSuppt{0-1} 

X  whatprocessnone{0-1} 

whatproductscustom{0-1 } 
whatproductsCOTS{0-1} 
whatproductscommoncust{0-1} 
whatproductsnone{0-1} 


Step  History 

Step  Parameter 

1  whattypeshrinkutilities{0-1} 

2  whatprocessSWEngSuppt{0-1} 

3  whatprocessappsuppt{0-1} 

4  whattypesystemsavionics{0-1} 

5  whatprocessnone{0-1} 

6  whattypecomponentdevelopment{0-1} 


Action 

Entered 

Entered 

Entered 

Entered 

Entered 

Entered 


Estimate  nDF 

6.80604058  1 

-0.8344756  1 

0  1 

0  1 

0  1 

0  1 

-0.9314213  1 

0  1 

0  1 

0  1 

0  1 

0  1 

-0.3990426  1 

0  1 

0  1 

0  1 

0  1 

0  1 

0  1 

0  1 

0  1 

0  1 

-0.391033  1 

0  1 

0  1 

0  1 

0  1 

0  1 

0  1 

0.41545638  1 

-0.8344756  1 

0  1 

0  1 

0  1 

0  1 


SS 

0 

5.06396 

0.486053 

0.828459 

0.139145 

0.171494 

8.929364 

1.961322 

0.038458 

0.370039 

1.020943 

0.768703 

3.452095 

0.307692 

0.004374 

0.467742 

0.196379 

1.124333 

0.348402 

0.751482 

0.196709 

0.072783 

4.43294 

0.007168 

1.662861 

0.847672 

0.168414 

1.114467 

0.002086 

4.785149 

5.06396 

0.00178 

0.019466 

0.061345 

0.590495 


"F  Ratio" 
0.000 
3.165 
0.298 
0.511 
0.085 
0.105 
5.581 
1.233 
0.023 
0.226 
0.632 
0.474 
2.157 
0.188 
0.003 
0.287 
0.120 
0.697 
0.213 
0.463 
0.120 
0.044 
2.770 
0.004 
1.040 
0.523 
0.103 
0.691 
0.001 
2.991 
3.165 
0.001 
0.012 
0.037 
0.363 


'Prob>F" 

1.0000 

0.0835 

0.5885 

0.4794 

0.7726 

0.7483 

0.0235 

0.2741 

0.8793 

0.6371 

0.4319 

0.4958 

0.1503 

0.6671 

0.9592 

0.5956 

0.7313 

0.4093 

0.6471 

0.5007 

0.7310 

0.8345 

0.1045 

0.9477 

0.3145 

0.4743 

0.7505 

0.4114 

0.9718 

0.0921 

0.0835 

0.9739 

0.9140 

0.8479 

0.5508 


'Sig  Prob"  Seq  SS 

0.0646  6.882668 

0.0690  6.294147 

0.1627  3.560905 

0.1564  3.579251 

0.1193  4.187049 

0.1503  3.452095 


RSquare  Cp  p 

0.0790  -8.29  2 

0.1512  -8.776  3 

0.1920  -8.182  4 

0.2331  -7.596  5 

0.281 1  -7.25  6 

0.3207  -6.614  7 


Appendix  B 


Response  Column  59 
Actual  by  Predicted  Plot 

8 
7 

1  6 

<5 

05 

in  4 

c 

|  3 
8  2 
1 
0 

1  2  3  4  5  6  7 

Column  59  Predicted  P=0.0199  RSq=0.32 
RMSE=1.2649 


Summary  of  Fit 


RSquare  0.320748 

RSquare  Adj  0.210599 

Root  Mean  Square  Error  1 .264943 

Mean  of  Response  4.204545 

Observations  (or  Sum  Wgts)  44 


Analysis  of  Variance 

Source 

DF 

Sum  of  Squares 

Mean  Square 

Model 

6 

27.956115 

4.65935 

Error 

37 

59.202975 

1 .60008 

C.  Total 

43 

87.159091 

F  Ratio 
2.9119 
Prob  >  F 
0.0199 


Lack  Of  Fit 


Source  DF 

Lack  Of  Fit  5 

Pure  Error  32 

Total  Error  37 


Sum  of  Squares 
11.932387 
47.270588 
59.202975 


Mean  Square 
2.38648 
1.47721 


F  Ratio 
1.6155 
Prob  >  F 
0.1842 
Max  RSq 
0.4577 


Parameter  Estimates 

Term 

Intercept 

whattypesystemsavionics[1  -0] 
whattypeshrinkutilities[1  -0] 
whattypecomponentdevelopment[1-0] 
whatprocessappsuppt[1-0] 
whatprocessSWEngSuppt[1  -0] 
whatprocessnone[1  -0] 


Estimate 

3.8310487 

1.6689513 

1.8628427 

0.7980853 

0.7820659 

-0.830913 

1.6689513 


Std  Error 
0.282973 
0.938144 
0.788564 
0.543349 
0.46986 
0.480484 
0.938144 


t  Ratio  Prob>|t| 

13.54  <.0001 

1.78  0.0835 

2.36  0.0235 

1.47  0.1503 

1.66  0.1045 

-1.73  0.0921 

1.78  0.0835 


Effect  Tests 

Source 

whattypesystemsavionics 

whattypeshrinkutilities 

whattypecomponentdevelopment 

whatprocessappsuppt 

whatprocessSWEngSuppt 

whatprocessnone 


Nparm  DF 

1  1 

1  1 

1  1 

1  1 

1  1 

1  1 


Sum  of  Squares 
5.0639599 
8.9293641 
3.4520953 
4.4329402 
4.7851490 
5.0639599 


F  Ratio  Prob  >  F 

3.1648  0.0835 

5.5806  0.0235 

2.1575  0.1503 

2.7704  0.1045 

2.9906  0.0921 

3.1648  0.0835 


Rework  =  3.83  +  sys-avia(1.67)  +  shrink-util(1 .86)  +  comp-dev(0.80)  +  proc-appsup(0.78)  +  proc-SWEngSup(-0.83)  + 
proc-none(1.67) 
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Stepwise  Fit  -  New  Survey  Data  -  Consequences  (Visibility) 

Response: 

Column  59 

Stepwise  Regression  Control 


Prob  to  Enter  0.250 

Prob  to  Leave  0.250 

Current  Estimates 

SSE  DFE  MSE  RSquare 

10.075292  23  0.4380562  0.8702 

Lock  Entered  Parameter 

X  X  Intercept 

X  whattypesystemsavionics{0-1} 

X  whattypesystemsembedded{0-1} 

X  whattypesystemscommunications{0-1 } 

X  whattypesystemsdevice{0-1} 

whattypeshrinkbusiness{0-1} 

X  whattypeshrinkutilities{0-1} 

whattypeshrinkinternet{0-1 } 
whattypecomponentdomain{0-1} 

X  whattypecomponentCASE{0-1} 

whattypecomponentclass{0- 1 } 

X  whattypecomponentOS{0-1} 

X  whattypecomponentdevelopment{0-1} 

whattypeenterpriseacctng{0-1 } 
whattypeenterprisemanufact{0-1 } 
whattypeenterprisepayroll{0-1 } 

X  whattypeenterprisescripting{0-1} 

whatprocessrequirements{0-1} 
whatprocessdesign{0-1} 

X  whatprocesstesting{0-1} 

whatprocessmaintenance{0-1} 

X  whatprocessreengineering{0-1} 

whatprocessappsuppt{0-1 } 
whatprocesstraining{0-1} 

X  whatprocessspecification{0-1} 

X  whatprocesscoding{0-1} 

whatprocessfielding{0-1} 

X  whatprocessCM{0-1} 

whatprocesstoolsuppt{0-1} 

X  whatprocessSWEngSuppt{0-1} 

X  whatprocessnone{0-1} 

X  whatproductscustom{0-1} 

whatproductsCOTS{0-1} 

X  whatproductscommoncust{0-1} 

X  whatproductsnone{0-1} 

Step  History 

Step  Parameter 


10 

11 

12 

13 

14 

15 

16 


24 

25 

26 

27 

28 
29 


whatproductsCOTS{0- 1 } 
whattypeenterprisescripting{0-1 } 
whattypecomponentdevelopment{0- 1 } 
whattypeenterprisepayroll{0-1} 
whattypecomponentdomain{0-1 } 
whattypesystemsdevice{0-1} 
whatprocessappsuppt{0-1 } 
whatprocesstoolsuppt{0-1} 
whattypesystemscommunications{0-1} 
whatproductsnone{0-1 } 
whattypecomponentdomain{0-1 } 
whatprocesstesti  ng{0- 1 } 
whatprocesscoding{0-1} 
whattypeenterprisepayroll{0-1} 
whatprocessnone{0-1 } 
whattypecomponentOS{0-1 } 
whatprocessSWEngSuppt{0-1} 
whatprocessCM{0-1} 
whatproductscustom{0-1 } 
whatproductsCOTS{0-1} 
whatprocessreengineering{0-1 } 
whatprocesstoolsuppt{0-1} 
whattypeshrinkutilities{0-1} 
whatprocessspecification{0-1} 
whatprocessappsuppt{0-1 } 
whattypesystemsavionics{0-1 } 
whatproductscommoncust{0-1} 
whattypesystemsembedded{0-1 } 
whattypecomponentCASE{0-1} 


RSquare  Adj  Cp  AIC 

0.7630  15.946415  -22.3979 


Estimate  nDF 

2.8495261  1 

0.37576496  1 

-0.4488408  1 

-0.7956194  1 

-1 .4740253  1 

0  1 

1.157259  1 

0  1 

0  1 

0.56574922  1 

0  1 

1.19102823  1 

-0.8669342  1 

0  1 

0  1 

0  1 

0.65175764  1 

0  1 

0  1 

-0.5517514  1 

0  1 

0.68668548  1 

0  1 

0  1 

-0.4210337  1 

1.13663035  1 

0  1 

0.66987347  1 

0  1 

-1.008367  1 

1.82147225  1 

-0.202901 1  1 

0  1 

0.29904591  1 

-1.5076248  1 


SS 

0 

0.660447 

1.189113 

9.364429 

17.62745 

0.005004 

5.663079 

0.017191 

0.217795 

0.747828 

0.059492 

9.044565 

9.023453 

0.015143 

0.14704 

0.174212 

2.443385 

0.287073 

0.041276 

5.247883 

0.021089 

8.36051 

0.067557 

0.006691 

3.679885 

15.40182 

0.01035 

3.521887 

0.101207 

6.462242 

12.84536 

0.627774 

0.236821 

1.1263 

14.298 


"F  Ratio"  "Prob>F" 

0.000  1.0000 

1.508  0.2319 

2.715  0.1130 

21.377  0.0001 

40.240  0.0000 

0.011  0.9177 

12.928  0.0015 

0.038  0.8480 

0.486  0.4930 

1.707  0.2043 

0.131  0.7212 

20.647  0.0001 

20.599  0.0001 

0.033  0.8573 

0.326  0.5739 

0.387  0.5402 

5.578  0.0270 

0.645  0.4304 

0.091  0.7664 

11.980  0.0021 

0.046  0.8319 

19.085  0.0002 

0.149  0.7037 

0.015  0.9049 

8.400  0.0081 

35.159  0.0000 

0.023  0.8818 

8.040  0.0094 

0.223  0.6412 

14.752  0.0008 

29.324  0.0000 

1.433  0.2435 

0.530  0.4745 

2.571  0.1225 

32.640  0.0000 


Action 

Entered 

Entered 

Entered 

Entered 

Entered 

Entered 

Entered 

Entered 

Entered 

Entered 

Removed 

Entered 

Entered 

Removed 

Entered 

Entered 

Entered 

Entered 

Entered 

Removed 

Entered 

Removed 

Entered 

Entered 

Removed 

Entered 

Entered 

Entered 

Entered 


"Sig  Prob"  Seq  SS 

0.0289  8.631175 

0.1388  3.722426 

0.0674  5.430911 

0.1032  4.089892 

0.1296  3.399955 

0.1880  2.494245 

0.1186  3.397585 

0.1815  2.411337 

0.2307  1.904735 

0.2128  2.025781 

0.2774  1.53092 

0.1021  3.387171 

0.0984  3.27593 

0.2658  1.449624 

0.0136  6.591409 

0.0538  3.533347 

0.0927  2.483923 

0.1040  2.183224 

0.1143  1.943052 

0.4063  0.519293 

0.0431  2.892784 

0.2673  0.823332 

0.1094  1.672484 

0.0578  2.167999 

0.4404  0.337895 

0.0599  1.897838 

0.1454  1.056782 

0.1772  0.871866 

0.2043  0.747828 


RSquare  Cp  p 

0.1112  90.747  2 

0.1591  85.747  3 

0.2291  77.534  4 

0.2818  71.844  5 

0.3256  67.45  6 

0.3577  64.76  7 

0.4015  60.37  8 

0.4325  57.836  9 

0.4571  56.254  10 

0.4832  54.445  11 

0.4635  55.324  10 

0.5071  50.954  11 

0.5493  46.794  12 

0.5306  47.52  1 1 

0.6155  37.125  12 

0.6610  32.48  13 

0.6930  29.809  14 

0.7212  27.704  15 

0.7462  26.05  16 

0.7395  25.026  15 

0.7768  21.587  16 

0.7662  21.135  15 

0.7877  19.99  16 

0.8156  17.913  17 

0.8113  16.548  16 

0.8357  14.979  17 

0.8493  14.992  18 

0.8606  15.353  19 

0.8702  15.946  20 


Appendix  B 


Response  Column  59 
Actual  by  Predicted  Plot 


Column  59  Predicted  P<.0001  RSq=0.87 
RMSE=0.6619 


Summary  of  Fit 


RSquare 

RSquare  Adj 

Root  Mean  Square  Error 

Mean  of  Response 

Observations  (or  Sum  Wgts) 

Analysis  of  Variance 

Source  DF  Sum 

Model  19 

Error  23 

C.  Total  42 

Parameter  Estimates 

Term 

Intercept 

whattypesystemsavionics[1  -0] 
whattypesystemsembedded[1  -0] 
whattypesystemscommunications[1  -0] 
whattypesystemsdevice[1  -0] 
whattypeshrinkutilities[1  -0] 
whattypecomponentCASE[1  -0] 
whattypecomponentOS[1  -0] 
whattypecomponentdevelopment[1-0] 
whattypeenterprisescripting[1  -0] 
whatprocesstesting[1  -0] 
whatprocessreengineering[1  -0] 
whatprocessspecification[1  -0] 
whatprocesscoding[1  -0] 
whatprocessCMjl  -0] 
whatprocessSWEngSuppt[1-0] 
whatprocessnone[1  -0] 
whatproductscustom[1-0] 
whatproductscommoncust[1-0] 
whatproductsnone[1-0] 

Effect  Tests 


0.87021 

0.762993 

0.661858 

4.093023 

43 


of  Squares  Mean  Square 

67.552615  3.55540 

10.075292  0.43806 

77.627907 


Estimate 

4.1276949 

-0.75153 

0.8976816 

1.5912387 

2.9480506 

-2.314518 

-1.131498 

-2.382056 

1.7338685 

-1.303515 

1.1035028 

-1.373371 

0.8420675 

-2.273261 

-1.339747 

2.0167339 

-3.642945 

0.4058023 

-0.598092 

3.0152496 


F  Ratio 
8.1163 
Prob  >  F 
<.0001 


Std  Error 
0.344614 
0.612057 
0.544848 
0.344159 
0.464735 
0.643723 
0.866 
0.524231 
0.382028 
0.551931 
0.318821 
0.314367 
0.290532 
0.383379 
0.472498 
0.525076 
0.672735 
0.338983 
0.372997 
0.527777 


t  Ratio  Prob>|t| 

11.98  <0001 

-1.23  0.2319 

1.65  0.1130 

4.62  0.0001 

6.34  <.0001 

-3.60  0.0015 

-1.31  0.2043 

-4.54  0.0001 

4.54  0.0001 

-2.36  0.0270 

3.46  0.0021 

-4.37  0.0002 

2.90  0.0081 

-5.93  <.0001 

-2.84  0.0094 

3.84  0.0008 

-5.42  <.0001 

1.20  0.2435 

-1.60  0.1225 

5.71  <.0001 


Source 

Nparm 

DF 

Sum  of  Squares 

F  Ratio 

Prob  >  F 

whattypesystemsavionics 

1 

1 

0.660447 

1 .5077 

0.2319 

whattypesystemsembedded 

1 

1 

1.189113 

2.7145 

0.1130 

whattypesystemscommunications 

1 

1 

9.364429 

21.3772 

0.0001 

whattypesystemsdevice 

1 

1 

17.627454 

40.2402 

<.0001 

whattypeshrinkutilities 

1 

1 

5.663079 

12.9277 

0.0015 

whattypecomponentCASE 

1 

1 

0.747828 

1.7072 

0.2043 

whattypecomponentOS 

1 

1 

9.044565 

20.6470 

0.0001 

whattypecomponentdevelopment 

1 

1 

9.023453 

20.5988 

0.0001 

whattypeenterprisescripting 

1 

1 

2.443385 

5.5778 

0.0270 

whatprocesstesting 

1 

1 

5.247883 

11.9799 

0.0021 

whatprocessreengineering 

1 

1 

8.360510 

19.0855 

0.0002 

whatprocessspecification 

1 

1 

3.679885 

8.4005 

0.0081 

whatprocesscoding 

1 

1 

15.401819 

35.1595 

<.0001 

whatprocessCM 

1 

1 

3.521887 

8.0398 

0.0094 

whatprocessSWEngSuppt 

1 

1 

6.462242 

14.7521 

0.0008 

whatprocessnone 

1 

1 

12.845365 

29.3236 

<.0001 

whatproductscustom 

1 

1 

0.627774 

1.4331 

0.2435 

whatproductscommoncust 

1 

1 

1.126300 

2.5711 

0.1225 

whatprod  uctsnone 

1 

1 

14.297998 

32.6396 

<.0001 

Visibility  =  4.13  +  sys-avia(-0.75)  +  sys-embed(0.90)  +  sys-comm(1.59)  +  sys-dev(2.95)  +  shrink-util(-2.31)  +  comp- 
CASE(-1.13)  +  comp-OS  (-2. 38)  +  comp-dev(1.73)  +  ent-script(-1.30)  +  proc-test(I.IO)  +  proc-reeng(-1 .37)  +  proc- 
spec(0.84)  +  proc-coding(-2.27)  +  proc-CM(-1.34)  +  proc-SWEngSup(2.02)  +  proc-none(-3.64)  +  prod-cust(0.41)  +  prod- 
comcust(-0.60)  +  prod-none(3.02) 
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Stepwise  Fit  -  New  Survey  Data  -  Consequences  (ControlProduct) 

Response: 

Column  59 


Stepwise  Regression  Control 

Prob  to  Enter  0.250 

Prob  to  Leave  0.250 

Direction: 

Rules: 


Current  Estimates 

SSE  DFE 

37.812636  38 


MSE  RSquare 

0.9950694  0.4771 


RSquare  Adj 
0.3945 


Cp  AIC 

-10.09682  6.169138 


Lock  Entered  Parameter 

X  X  Intercept 

whattypesystemsavionics{0-1 } 
whattypesystemsembedded{0-1 } 
whattypesystemscommunications{0-1} 
whattypesystemsdevice{0- 1 } 
whattypeshrinkbusiness{0-1} 
whattypeshrinkutilities{0-1 } 
whattypeshrinkinternet{0-1 } 
whattypecomponentdomain{0-1} 
whattypecomponentCASE{0-1} 
whattypecomponentclass{0- 1 } 
whattypecomponentOS{0-1 } 
whattypecomponentdevelopment{0-1} 
whattypeenterpriseacctng{0-1 } 
whattypeenterprisemanufact{0-1} 
whattypeenterprisepayroll{0-1 } 
whattypeenterprisescripting{0-1} 

X  whatprocessrequirements{0-1} 

whatprocessdesign{0- 1 } 

X  whatprocesstesting{0-1} 

whatprocessmaintenance{0-1} 
whatprocessreengineering{0-1 } 
whatprocessappsuppt{0-1 } 

X  whatprocesstraining{0-1} 

whatprocessspecification{0-1 } 
whatprocesscoding{0- 1 } 

X  whatprocessfielding{0-1} 

whatprocessCM{0-1} 
whatprocesstoolsuppt{0-1} 

X  whatprocessSWEngSuppt{0-1} 

whatprocessnone{0-1 } 
whatproductscustom{0-1 } 
whatproductsCOTS{0-1} 
whatproductscommoncust{0-1} 
whatproductsnone{0-1} 


Step  History 

Step  Parameter 

1  whatprocessrequirements{0-1} 

2  whatproductsnone{0-1} 

3  whatprocessfielding{0-1} 

4  whatprocessSWEngSuppt{0-1} 

5  whatprocesstesting{0-1} 

6  whatprocesstraining{0-1} 


Action 

Entered 

Entered 

Entered 

Entered 

Entered 

Entered 


Estimate 

4.4580914 

0 

0 

0 

0 

0 

0 


nDF 

1 

1 

1 

1 

1 

1 

1 


0 

0 

0 

0 

-0.6446867 

0 

0.25381386 


1.20324177 

0 

0 

-0.4738513 

0 

0 

0 

0 

-0.8482748 


"Sig  Prob" 
0.0255 
0.0299 
0.0081 
0.0194 
0.1515 
0.1619 


Seq  SS 
8.011542 
6.903736 
9.132481 
6.239641 
2.185861 
2.025214 


SS 

0 

0.25488 

0.051053 

0.82343 

0.177164 

0.269405 

1.235669 

0.671233 

0.458799 

0.003958 

0.532813 

0.271947 

0.001461 

0.121053 

0.036758 

0.00179 

0.309581 

10.75372 

0.147606 

2.176322 

0.109231 

0.422917 

0.191507 

2.025214 

0.00596 

0.289823 

14.90044 

0.136947 

0.043319 

5.33075 

0.548686 

0.063472 

0.613553 

0.011881 

10.47251 


RSquare 

0.1108 

0.2063 

0.3326 

0.4188 

0.4491 

0.4771 


"F  Ratio"  "Prob>F" 

0.000  1.0000 

0.251  0.6193 

0.050  0.8243 

0.824  0.3700 

0.174  0.6788 

0.266  0.6094 

1.250  0.2708 

0.669  0.4187 

0.454  0.5044 

0.004  0.9507 

0.529  0.4717 

0.268  0.6077 

0.001  0.9700 

0.119  0.7323 

0.036  0.8505 

0.002  0.9668 

0.305  0.5838 

10.807  0.0022 

0.145  0.7055 

2.187  0.1474 

0.107  0.7452 

0.419  0.5217 

0.188  0.6668 

2.035  0.1619 

0.006  0.9395 

0.286  0.5961 

14.974  0.0004 

0.134  0.7159 

0.042  0.8379 

5.357  0.0261 

0.545  0.4651 

0.062  0.8044 

0.610  0.4397 

0.012  0.9147 

10.524  0.0025 


Cp  p 

-5.455  2 

-7.271  3 

-10.32  4 

-11.77  5 

-10.98  6 

-10.1  7 
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Response  Column  59 
Actual  by  Predicted  Plot 


Column  59  Predicted  P=0.0002  RSq=0.48 
RMSE=0.9975 


Summary  of  Fit 


RSquare 

0.477084 

RSquare  Adj 

0.394518 

Root  Mean  Square  Error 

0.997532 

Mean  of  Response 

4.244444 

Observations  (or  Sum  Wgts) 

45 

Analysis  of  Variance 

Source 

DF 

Sum  of  Squares 

Mean  Square 

F  Ratio 

Model 

6 

34.498475 

5.74975 

5.7782 

Error 

38 

37.812636 

0.99507 

Prob  >  F 

C.  Total 

44 

72.311111 

0.0002 

Lack  Of  Fit 

Source 

DF 

Sum  of  Squares 

Mean  Square 

F  Ratio 

Lack  Of  Fit 

10 

10.667398 

1.06674 

1.1003 

Pure  Error 

28 

27.145238 

0.96947 

Prob  >  F 

Total  Error 

38 

37.812636 

0.3955 

Max  RSq 
0.6246 


Parameter  Estimates 


Term 

Estimate 

Std  Error 

t  Ratio  Prob>|t| 

Intercept 

3.6967322 

0.278474 

13.27  <.0001 

whatprocessrequirements[1-0] 

1.2893735 

0.392217 

3.29  0.0022 

whatprocesstesting[1  -0] 

-0.507628 

0.34325 

-1.48  0.1474 

whatprocesstraining[1-0] 

0.503204 

0.352724 

1.43  0.1619 

whatprocessfieldingjl  -0] 

-2.406484 

0.621885 

-3.87  0.0004 

whatprocessSWEngSuppt[1  -0] 

0.9477026 

0.409454 

2.31  0.0261 

whatproductsnone[1-0] 

1.6965495 

0.522959 

3.24  0.0025 

Effect  Tests 

Source 

Nparm  DF 

Sum  of  Squares 

F  Ratio 

Prob  >  F 

whatprocessrequirements 

1  1 

10.753721 

10.8070 

0.0022 

whatprocesstesting 

1  1 

2.176322 

2.1871 

0.1474 

whatprocesstraining 

1  1 

2.025214 

2.0352 

0.1619 

whatprocessfielding 

1  1 

14.900445 

14.9743 

0.0004 

whatprocessSWEngSuppt 

1  1 

5.330750 

5.3572 

0.0261 

whatproductsnone 

1  1 

10.472506 

10.5244 

0.0025 

ControlProduct  =  3.70  +  proc-req(1.29)  +  proc-test(-0.51)  +  proc-train(0.50)  +  proc-field(-2.41)  +  proc-SWEngSup(0.95)  + 
prod-none(1.70) 
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Stepwise  Fit  -  New  Survey  Data  -  Consequences  (ChangeCost) 

Response: 

Column  59 


Stepwise  Regression  Control 

Prob  to  Enter  0.250 

Prob  to  Leave  0.250 

Direction: 

Rules: 


Current  Estimates 

SSE  DFE 

69.585674  38 


MSE  RSquare 

1.831202  0.3630 


RSquare  Adj  Cp  AIC 

0.2625  -13.14053  33.61533 


Lock  Entered  Parameter 

X  X  Intercept 

whattypesystemsavionics{0-1 } 
whattypesystemsembedded{0-1 } 
whattypesystemscommunications{0-1} 
whattypesystemsdevice{0- 1 } 
whattypeshrinkbusiness{0-1} 

X  whattypeshrinkutilities{0-1} 

whattypeshrinkinternet{0-1 } 
whattypecomponentdomain{0-1} 
whattypecomponentCASE{0-1} 

X  whattypecomponentclass{0-1} 

whattypecomponentOS{0-1 } 
whattypecomponentdevelopment{0-1} 
whattypeenterpriseacctng{0-1 } 
whattypeenterprisemanufact{0-1} 
whattypeenterprisepayroll{0-1 } 
whattypeenterprisescripting{0-1} 
whatprocessrequirements{0-1 } 
whatprocessdesign{0- 1 } 
whatprocesstesting{0- 1 } 
whatprocessmaintenance{0-1} 
whatprocessreengineering{0-1} 
whatprocessappsuppt{0-1 } 
whatprocesstraining{0-1} 
whatprocessspecification{0-1 } 

X  whatprocesscoding{0-1} 

whatprocessfielding{0-1} 
whatprocessCM{0-1} 
whatprocesstoolsuppt{0-1} 
whatprocessSWEngSuppt{0- 1 } 

X  whatprocessnone{0-1} 

whatproductscustom{0-1 } 
whatproductsCOTS{0-1} 

X  whatproductscommoncust{0-1} 

whatproductsnone{0-1} 


Step  History 

Step  Parameter 

1  whatproductsnone{0-1} 

2  whatprocesscoding{0-1} 

3  whattypecomponentclass{0-1} 

4  whatproductscommoncust{0-1} 

5  whattypeshrinkutilities{0-1} 

6  whatprocessnone{0-1} 


Action 

Entered 

Entered 

Entered 

Entered 

Entered 

Entered 


Estimate  nDF 

3.89054647  1 

0  1 

0  1 

0  1 

0  1 

0  1 

-0.6274136  1 

0  1 

0  1 

0  1 

1.13712109  1 

0  1 

0  1 

0  1 

0  1 

0  1 

0  1 

0  1 

0  1 

0  1 

0  1 

0  1 

0  1 

0  1 

0  1 

0.35740359  1 

0  1 

0  1 

0  1 

0  1 

-0.9283976  1 

0  1 

0  1 

-0.6468021  1 

1.03925303  1 


"Sig  Prob" 
0.0948 
0.0514 
0.0471 
0.0605 
0.1514 
0.1549 


Seq  SS 
6.944444 
8.942202 
8.656957 
7.224869 
4.033188 
3.85711 


SS 

0 

0.040887 

0.907083 

0.489532 

0.276763 

0.037681 

4.234228 

1.522923 

1.287347 

1.041879 

12.74785 

0.001035 

0.608705 

0.016719 

0.598182 

0.456885 

0.557332 

0.011286 

0.144105 

0.950362 

1.062279 

0.53588 

I. 81904 
1.026726 
1.964361 
3.752321 
0.000325 
0.102754 
0.003435 
1.141334 

3.85711 

0.076272 

0.173983 

9.117309 

II. 6174 


RSquare 

0.0636 

0.1454 

0.2247 

0.2908 

0.3277 

0.3630 


"F  Ratio" 
0.000 
0.022 
0.489 
0.262 
0.148 
0.020 
2.312 
0.828 
0.697 
0.562 
6.961 
0.001 
0.327 
0.009 
0.321 
0.245 
0.299 
0.006 
0.077 
0.512 
0.574 
0.287 
0.993 
0.554 
1.075 
2.049 
0.000 
0.055 
0.002 
0.617 
2.106 
0.041 
0.093 
4.979 
6.344 


Cp 

-14.74 

-15.04 

-15.26 

-15.12 

-14.15 

-13.14 


"Prob>F" 

1.0000 

0.8835 

0.4889 

0.6117 

0.7029 

0.8882 

0.1366 

0.3688 

0.4090 

0.4580 

0.0120 

0.9814 

0.5712 

0.9254 

0.5745 

0.6239 

0.5880 

0.9387 

0.7832 

0.4786 

0.4536 

0.5953 

0.3254 

0.4613 

0.3066 

0.1605 

0.9896 

0.8163 

0.9661 

0.4372 

0.1549 

0.8414 

0.7624 

0.0316 

0.0161 

P 

2 

3 

4 

5 

6 

7 
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Response  Column  59 
Actual  by  Predicted  Plot 

8 
7 
ro  6 
<5 

CT) 


!  3 
O  2 
1 
0 

1  2  3  4  5  6  7 

Column  59  Predicted  P=0.0063  RSq=0.36 
RMSE=1 .3532 


Summary  of  Fit 


RSquare  0.363028 

RSquare  Adj  0.262453 

Root  Mean  Square  Error  1.353219 

Mean  of  Response  3.711111 

Observations  (or  Sum  Wgts)  45 


Analysis  of  Variance 


Source  DF 

Model  6 

Error  38 

C.  Total  44 


Sum  of  Squares 
39.65877 
69.58567 
109.24444 


Mean  Square 
6.60980 
1.83120 


F  Ratio 
3.6095 
Prob  >  F 
0.0063 


Lack  Of  Fit 


Source  DF 

Lack  Of  Fit  5 

Pure  Error  33 

Total  Error  38 


Sum  of  Squares  Mean  Square 

15.650892  3.13018 

53.934783  1.63439 

69.585674 


F  Ratio 
1.9152 
Prob  >  F 
0.1184 
Max  RSq 
0.5063 


Parameter  Estimates 


Term 

Estimate 

Std  Error 

t  Ratio  Prob>|t| 

Intercept 

4.2217108 

0.485034 

8.70  <.0001 

whattypeshrinkutilities[1  -0] 

1.2548273 

0.825211 

1.52  0.1366 

whattypecomponentclass[1  -0] 

-2.274242 

0.861958 

-2.64  0.0120 

whatprocesscoding[1  -0] 

-0.714807 

0.499352 

-1.43  0.1605 

whatprocessnone[1  -0] 

1 .8567952 

1.279385 

1.45  0.1549 

whatproductscommoncust[1-0] 

1.2936042 

0.579744 

2.23  0.0316 

whatproductsnone[1-0] 

-2.078506 

0.825211 

-2.52  0.0161 

Effect  Tests 

Source 

Nparm  DF 

Sum  of  Squares 

F  Ratio 

Prob  >  F 

whattypeshrinkutilities 

1  1 

4.234228 

2.3123 

0.1366 

whattypecomponentclass 

1  1 

12.747855 

6.9615 

0.0120 

whatprocesscoding 

1  1 

3.752321 

2.0491 

0.1605 

whatprocessnone 

1  1 

3.857110 

2.1063 

0.1549 

whatproductscommoncust 

1  1 

9.117309 

4.9789 

0.0316 

whatproductsnone 

1  1 

11.617399 

6.3441 

0.0161 

ChangeCost  =  4.22  +  shrink-util(1.25)  +  comp-class(-2.27)  +  proc-coding(-0.71)  +  proc-none(1.86)  +  prod-comcust(1.29) 
+  prod-none(-2.08) 
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Stepwise  Fit  -  New  Survey  Data  -  Consequences  (LangCult) 

Response: 

Column  59 


Stepwise  Regression  Control 

Prob  to  Enter  0.250 

Prob  to  Leave  0.250 

Direction: 

Rules: 


Current  Estimates 

SSE  DFE 

22.10621  29 

Lock  Entered  Parameter 

X  X  Intercept 


MSE  RSquare  RSquare  Adj  Cp 

0.7622831  0.6052  0.4147  -0.405498 

Estimate  nDF 

4.32912166  1 

0  1 

-1.227754  1 

0  1 

0  1 

0  1 

0.83896766  1 


whattypesystemsavionics{0-1 } 

whattypesystemsembedded{0-1 } 

whattypesystemscommunications{0-1} 

whattypesystemsdevice{0- 1 } 

whattypeshrinkbusiness{0-1} 

whattypeshrinkutilities{0-1} 

whattypeshrinkinternet{0-1 } 

whattypecomponentdomain{0-1} 

whattypecomponentCASE{0-1} 

whattypecomponentclass{0- 1 } 

whattypecomponentOS{0-1 } 

whattypecomponentdevelopment{0-1} 

whattypeenterpriseacctng{0-1} 

whattypeenterprisemanufact{0-1} 

whattypeenterprisepayroll{0-1} 

whattypeenterprisescripting{0-1} 

whatprocessrequirements{0-1} 

whatprocessdesign{0- 1 } 

whatprocesstesting{0- 1 } 

whatprocessmaintenance{0-1} 

whatprocessreengineering{0-1} 

whatprocessappsuppt{0-1 } 

whatprocesstraining{0-1} 

whatprocessspecification{0-1 } 

whatprocesscoding{0- 1 } 

whatprocessfielding{0-1} 

whatprocessCM{0-1} 

whatprocesstoolsuppt{0-1} 

whatprocessSWEngSuppt{0- 1 } 

whatprocessnone{0-1 } 

whatproductscustom{0-1 } 

whatproductsCOTS{0-1} 

whatproductscommoncust{0-1} 

whatproductsnone{0-1} 


Step  History 

Step  Parameter 

1  whatprocessdesign{0-1} 

2  whatprocesstraining{0-1} 

3  whatproductsnone{0-1} 

4  whattypeenterprisepayroll{0-1} 

5  whattypecomponentOS{0-1} 

6  whattypecomponentclass{0-1} 

7  whatprocesstraining{0-1} 

8  whatprocessspecification{0-1} 

9  whattypesystemsembedded{0-1} 

10  whatprocessappsuppt{0-1} 

1 1  whattypecomponentdomain{0-1 } 

12  whattypeshrinkutilities{0-1} 

13  whatprocessCM{0-1} 

14  whatproductscustom{0-1} 

15  whattypecomponentclass{0-1} 

16  whatprocessreengineering{0-1} 

17  whatprocessfielding{0-1} 

18  whattypecomponentclass{0-1} 


Action 

Entered 

Entered 

Entered 

Entered 

Entered 

Entered 

Removed 

Entered 

Entered 

Entered 

Entered 

Entered 

Entered 

Entered 

Removed 

Entered 

Entered 

Entered 


-0.6498443 

0 

0.45953112 

1.07071857 

0 

0 

0 

-0.8745822 

0 

0 

0.78951074 


0.41116451 

0.53615387 

0 

-0.6586279 

0 

-0.5010646 

-0.4352561 

0 

0 

0 

0.31992256 

0 

0 

0.68658062 

"Sig  Prob" 
0.0611 
0.1076 
0.1824 
0.1673 
0.1182 
0.0980 
0.3299 
0.1978 
0.1575 
0.2070 
0.1841 
0.0760 
0.1559 
0.1890 
0.2642 
0.1771 
0.1696 
0.1525 


1 


Seq  SS 
4.538947 
3.185832 
2.124874 
2.230158 
2.76754 
2.9736 
1.005952 
1.740741 
2.048578 
1.596211 
1.733516 
2.960754 
1.802133 
1.501058 
1.076403 
1.558419 
1.568319 
1 .645464 


AIC 

-0.28657 

SS 

0 

0.16874 
10.0837 
0.109937 
0.026113 
0.041507 
5.061568 
0.396095 
3.579982 
0.432215 
1 .645464 
9.56651 1 
0.062519 
0.225621 
0.25625 
7.322478 
0.094514 
0.051091 
12.30375 
0.012155 
0.528652 
3.115304 
6.165648 
0.668773 
7.039915 
0.63947 
2.178171 
3.562696 
0.0677 
0.54531 
0.308893 
1 .99771 
0.328891 
0.000054 
5.165632 

RSquare 
0.0811 
0.1379 
0.1759 
0.2157 
0.2651 
0.3182 
0.3003 
0.3314 
0.3679 
0.3964 
0.4274 
0.4803 
0.5124 
0.5392 
0.5200 
0.5479 
0.5759 
0.6052 


"F  Ratio" 
0.000 
0.215 
13.228 
0.140 
0.033 
0.053 
6.640 
0.511 
4.696 
0.558 
2.159 
12.550 
0.079 
0.289 
0.328 
9.606 
0.120 
0.065 
16.141 
0.015 
0.686 
4.087 
8.088 
0.874 
9.235 
0.834 
2.857 
4.674 
0.086 
0.708 
0.397 
2.621 
0.423 
0.000 
6.777 


Cp 

-8.353 

-8.313 

-7.619 

-6.991 

-6.693 

-6.521 

-7.903 

-6.973 

-6.233 

-5.215 

-4.281 

-4.101 

-3.21 

-2.133 

-3.471 

-2.429 

-1.394 

-0.405 


'Prob>F" 

1.0000 

0.6462 

0.0011 

0.7112 

0.8569 

0.8201 

0.0153 

0.4807 

0.0386 

0.4611 

0.1525 

0.0014 

0.7802 

0.5953 

0.5712 

0.0043 

0.7314 

0.8008 

0.0004 

0.9021 

0.4145 

0.0525 

0.0081 

0.3580 

0.0050 

0.3689 

0.1017 

0.0390 

0.7715 

0.4072 

0.5339 

0.1163 

0.5208 

0.9935 

0.0144 


P 

2 

3 

4 

5 

6 
7 
6 

7 

8 
9 

10 

11 

12 

13 

12 

13 

14 

15 
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Response  Column  59 
Actual  by  Predicted  Plot 


TO  c 

3  5 

OT  4 
m 

|3 

o  o 
O  * 


1  2  3  4  5  6  7 

Column  59  Predicted  P=0.0042  RSq=0.61 
RMSE=0.8731 


Summary  of  Fit 


RSquare  0.605246 

RSquare  Adj  0.414675 

Root  Mean  Square  Error  0.873088 

Mean  of  Response  4 

Observations  (or  Sum  Wgts)  44 


Analysis  of  Variance 

Source  DF  Sum  of  Squares 

Model  14  33.893790 

Error  29  22.106210 

C.  Total  43  56.000000 


Mean  Square  F  Ratio 

2.42098  3.1760 

0.76228  Prob  >  F 

0.0042 


Parameter  Estimates 

Term 

Intercept 

whattypesystemsembedded[1  -0] 
whattypeshrinkutilities[1  -0] 
whattypecomponentdomain[1  -0] 
whattypecomponentclass[1  -0] 
whattypecomponentOS[1  -0] 
whattypeenterprisepayroll[1-0] 
whatprocessdesign[1  -0] 
whatprocessreengineering[1  -0] 
whatprocessappsuppt[1  -0] 
whatprocessspecification[1  -0] 
whatprocessfielding[1  -0] 
whatprocessCM[1  -0] 
whatproductscustom[1-0] 
whatproductsnone[1-0] 


Estimate 
5.0945421 
2.455508 
-1.677935 
1.2996886 
-0.919062 
-2.141437 
1.7491645 
-1.579021 
-0.822329 
-1 .072308 
1.3172559 
1.0021292 
0.8705122 
-0.639845 
-1.373161 


Std  Error 
0.35379 
0.675133 
0.651165 
0.599732 
0.625545 
0.604487 
0.564365 
0.393032 
0.406774 
0.377041 
0.433456 
0.592838 
0.402665 
0.395245 
0.527495 


t  Ratio 
14.40 
3.64 
-2.58 
2.17 
-1.47 
-3.54 
3.10 
-4.02 
-2.02 
-2.84 
3.04 
1.69 
2.16 
-1.62 
-2.60 


Prob>|t| 

<.0001 

0.0011 

0.0153 

0.0386 

0.1525 

0.0014 

0.0043 

0.0004 

0.0525 

0.0081 

0.0050 

0.1017 

0.0390 

0.1163 

0.0144 


Effect  Tests 

Source 

whattypesystemsembedded 

whattypeshrinkutilities 

whattypecomponentdomain 

whattypecomponentclass 

whattypecomponentOS 

whattypeenterprisepayroll 

whatprocessdesign 

whatprocessreengineering 

whatprocessappsuppt 

whatprocessspecification 

whatprocessfielding 

whatprocessCM 

whatproductscustom 

whatproductsnone 


Nparm  DF 

1  1 

1  1 

1  1 

1  1 

1  1 

1  1 

1  1 

1  1 

1  1 

1  1 

1  1 

1  1 

1  1 

1  1 


Sum  of  Squares 
10.083700 
5.061568 
3.579982 
1 .645464 
9.56651 1 
7.322478 
12.303747 
3.115304 
6.165648 
7.039915 
2.178171 
3.562696 
1.997710 
5.165632 


F  Ratio  Prob  >  F 

13.2283  0.0011 

6.6400  0.0153 

4.6964  0.0386 

2.1586  0.1525 

12.5498  0.0014 

9.6060  0.0043 

16.1407  0.0004 

4.0868  0.0525 

8.0884  0.0081 

9.2353  0.0050 

2.8574  0.1017 

4.6737  0.0390 

2.6207  0.1163 

6.7765  0.0144 


LangCult  =  5.09  +  sys-embed(2.46)  +  shrink-util(-1 .68)  +  comp-domain(1.30)  +  comp-class(-0.92)  +  comp-OS(-2.14)  + 
ent-pay(1.75)  +  proc-des(-1.58)  +  proc-reeng(-0.82)  +  proc-appsup(-1 .07)  +  proc-spec(1.32)  +  proc-field(1 .00)  +  proc- 
CM(0.87)  +  prod-cust(-0.64)  +  prod-none(-1.37) 
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Stepwise  Fit  -  New  Survey  Data  -  Consequences  (TurfWar) 

Response: 

Column  59 


Stepwise  Regression  Control 

Prob  to  Enter  0.250 

Prob  to  Leave  0.250 

Direction: 

Rules: 


Current  Estimates 

SSE  DFE 

48.028323  35 


MSE  RSquare 

1.3722378  0.4834 


RSquare  Adj 
0.3506 


Cp  AIC 

-6.173772  22.93078 


Lock  Entered  Parameter 

X  X  Intercept 

whattypesystemsavionics{0-1 } 

X  whattypesystemsembedded{0-1} 

whattypesystemscommunications{0-1} 
X  whattypesystemsdevice{0-1} 

whattypeshrinkbusiness{0-1} 
whattypeshrinkutilities{0-1 } 

X  whattypeshrinkinternet{0-1} 

whattypecomponentdomain{0-1} 
whattypecomponentCASE{0-1} 
whattypecomponentclass{0- 1 } 
whattypecomponentOS{0-1 } 
whattypecomponentdevelopment{0-1} 
whattypeenterpriseacctng{0-1 } 
whattypeenterprisemanufact{0-1} 
whattypeenterprisepayroll{0-1 } 

X  whattypeenterprisescripting{0-1} 

whatprocessrequirements{0-1 } 

X  whatprocessdesign{0-1} 

whatprocesstesting{0- 1 } 
whatprocessmaintenance{0-1} 
whatprocessreengineering{0-1} 
whatprocessappsuppt{0-1 } 
whatprocesstraining{0-1} 
whatprocessspecification{0-1 } 
whatprocesscoding{0- 1 } 
whatprocessfielding{0-1} 

X  whatprocessCM{0-1} 

whatprocesstoolsuppt{0-1 } 

X  whatprocessSWEngSuppt{0-1} 

whatprocessnone{0-1 } 
whatproductscustom{0-1 } 
whatproductsCOTS{0-1} 

X  whatproductscommoncust{0-1} 

X  whatproductsnone{0-1} 

Step  History 

Step  Parameter 

1  whatprocessSWEngSuppt{0-1} 

2  whatproductscommoncust{0-1} 

3  whattypesystemsembedded{0-1} 

4  whattypeshrinkutilities{0-1} 

5  whatprocessCM{0-1} 

6  whatprocessdesign{0-1} 

7  whatproductsnone{0-1} 

8  whattypeenterprisescripting{0-1 } 

9  whattypesystemsdevice{0-1} 

10  whattypeshrinkinternet{0-1} 

11  whattypeshrinkutilities{0-1} 


Action 

Entered 

Entered 

Entered 

Entered 

Entered 

Entered 

Entered 

Entered 

Entered 

Entered 

Removed 


Estimate  nDF 

2.78177383  1 

0  1 

-0.716114  1 

0  1 

0.67807578  1 

0  1 

0  1 

0.36601421  1 

0  1 

0  1 

0  1 

0  1 

0  1 

0  1 

0  1 

0  1 

0.9089169  1 

0  1 

0.65364789  1 

0  1 

0  1 

0  1 

0  1 

0  1 

0  1 

0  1 

0  1 

-0.7383293  1 

0  1 

0.87281691  1 

0  1 

0  1 

0  1 

-0.4229876  1 

0.72215469  1 


SS 

0 

0.095973 

5.201008 

0.007599 

5.137561 

0.076437 

1.70838 

3.332137 

0.030299 

0.13776 

0.7849 

1.116724 

1.253262 

1.025431 

0.43564 

0.486418 

5.577608 

0.158059 

10.92475 

1.293867 

0.073074 

0.565592 

0.050432 

0.041659 

1.004417 

0.17254 

0.126291 

8.854045 

0.060059 

15.08296 

0.092105 

0.108546 

0.124588 

3.651643 

7.357193 


"Sig  Prob" 
0.0262 
0.0344 
0.0383 
0.1373 
0.1548 
0.1482 
0.1735 
0.1998 
0.1771 
0.1838 
0.2706 


Seq  SS  RSquare 

10.20134  0.1097 

8.460391  0.2007 

7.472752  0.2811 

3.633324  0.3202 

3.237583  0.3550 

3.251326  0.3900 

2.805935  0.4201 

2.439374  0.4464 

2.647484  0.4748 

2.50833  0.5018 

1 .70838  0.4834 


"F  Ratio" 
0.000 
0.068 
3.790 
0.005 
3.744 
0.054 
1.254 
2.428 
0.021 
0.098 
0.565 
0.809 
0.911 
0.742 
0.311 
0.348 
4.065 
0.112 
7.961 
0.941 
0.052 
0.405 
0.036 
0.030 
0.726 
0.123 
0.090 
6.452 
0.043 
10.992 
0.065 
0.077 
0.088 
2.661 
5.361 


Cp 

-8.553 

-9.869 

-10.8 

-10.22 

-9.492 

-8.766 

-7.866 

-6.822 

-5.86 

-4.843 

-6.174 


'Prob>F" 

1.0000 

0.7957 

0.0596 

0.9420 

0.0611 

0.8173 

0.2706 

0.1282 

0.8844 

0.7564 

0.4575 

0.3746 

0.3466 

0.3951 

0.5806 

0.5592 

0.0515 

0.7396 

0.0078 

0.3388 

0.8213 

0.5287 

0.8512 

0.8646 

0.4001 

0.7284 

0.7665 

0.0157 

0.8378 

0.0021 

0.7998 

0.7831 

0.7680 

0.1118 

0.0266 


P 

2 

3 

4 

5 

6 

7 

8 
9 

10 

11 

10 


Appendix  B 


Response  Column  59 
Actual  by  Predicted  Plot 

8 
7 
ro  6 

<  5 

CT> 

in  4 
c 

i  3 
O  2 
1 
0 

1  2  3  4  5  6  7 

Column  59  Predicted  P=0.0027  RSq=0.48 
RMSE=1.1714 


Summary  of  Fit 


RSquare  0.483443 

RSquare  Adj  0.350614 

Root  Mean  Square  Error  1.171426 

Mean  of  Response  3.977778 

Observations  (or  Sum  Wgts)  45 


Analysis  of  Variance 

Source  DF  Sum  of  Squares 

Model  9  44.949455 

Error  35  48.028323 

C.  Total  44  92.977778 


Mean  Square  F  Ratio 

4.99438  3.6396 

1.37224  Prob  >  F 

0.0027 


Lack  Of  Fit 

Source  DF  Sum  of  Squares 

Lack  Of  Fit  13  14.444990 

Pure  Error  22  33.583333 

Total  Error  35  48.028323 


Mean  Square  F  Ratio 

1.11115  0.7279 

1.52652  Prob  >  F 

0.7191 
Max  RSq 
0.6388 


Parameter  Estimates 

Term 

Intercept 

whattypesystemsembedded[1  -0] 
whattypesystemsdevice[1  -0] 
whattypeshrinkinternet[1  -0] 
whattypeenterprisescripting[1-0] 
whatprocessdesign[1  -0] 
whatprocessCM[1  -0] 
whatprocessSWEngSuppt[1  -0] 
whatproductscommoncust[1-0] 
whatproductsnone[1-0] 


Estimate 
5.1059692 
1.4322279 
-1.356152 
-0.732028 
-1.817834 
-1 .307296 
1.4766587 
-1.745634 
0.8459753 
-1.444309 


Std  Error 
0.423039 
0.73567 
0.700881 
0.469766 
0.901664 
0.463322 
0.581332 
0.526532 
0.518595 
0.623762 


t  Ratio 
12.07 
1.95 
-1.93 
-1.56 
-2.02 
-2.82 
2.54 
-3.32 
1.63 
-2.32 


Prob>|t| 

<.0001 

0.0596 

0.0611 

0.1282 

0.0515 

0.0078 

0.0157 

0.0021 

0.1118 

0.0266 


Effect  T ests 

Source 

whattypesystemsembedded 

whattypesystemsdevice 

whattypeshrinkinternet 

whattypeenterprisescripting 

whatprocessdesign 

whatprocessCM 

whatprocessSWEngSuppt 

whatproductscommoncust 

whatproductsnone 


Nparm  DF 

1  1 

1  1 

1  1 

1  1 

1  1 

1  1 

1  1 

1  1 

1  1 


Sum  of  Squares 
5.201008 
5.137561 
3.332137 
5.577608 
10.924751 
8.854045 
15.082964 
3.651643 
7.357193 


F  Ratio  Prob  >  F 

3.7902  0.0596 

3.7439  0.0611 

2.4283  0.1282 

4.0646  0.0515 

7.9613  0.0078 

6.4523  0.0157 

10.9915  0.0021 

2.6611  0.1118 

5.3615  0.0266 


TurfWar  =  5.11  +  sys-embed(1.43)  +  sys-dev(-1.36)  +  shrink-int(-0.73)  +  ent-script(-1 .82)  +  proc-des(-1.31)  +  proc- 
CM(1.48)  +  proc-SWEngSup(-1.75)  +  prod-comcust(0.85)  +  prod-none(-1 .44) 
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Stepwise  Fit  -  New  Survey  Data  -  Consequences  (FailLikely) 

Response: 

Column  59 


Stepwise  Regression  Control 

Prob  to  Enter  0.250 

Prob  to  Leave  0.250 

Direction: 

Rules: 


Current  Estimates 

SSE  DFE 

22.536245  30 


MSE  RSquare 

0.7512082  0.7548 


RSquare  Adj 
0.6404 


Cp  AIC 

-1.351474  -1.11919 


Lock 

X 


X  Intercept 

whattypesystemsavionics{0-1 } 
whattypesystemsembedded{0-1 } 
whattypesystemscommunications{0-1} 
whattypesystemsdevice{0-1 } 

X  whattypeshrinkbusiness{0-1} 

X  whattypeshrinkutilities{0-1} 

whattypeshrinkinternet{0-1 } 
whattypecomponentdomain{0-1} 
whattypecomponentCASE{0-1} 

X  whattypecomponentclass{0-1} 

whattypecomponentOS{0-1 } 
whattypecomponentdevelopment{0-1} 
X  whattypeenterpriseacctng{0-1} 

whattypeenterprisemanufact{0-1 } 
whattypeenterprisepayroll{0-1 } 

X  whattypeenterprisescripting{0-1 } 

X  whatprocessrequirements{0-1} 

whatprocessdesign{0- 1 } 

X  whatprocesstesting{0-1} 

X  whatprocessmaintenance{0-1} 

X  whatprocessreengineering{0-1} 

whatprocessappsuppt{0-1 } 
whatprocesstraining{0-1} 
whatprocessspecification{0-1 } 
whatprocesscoding{0- 1 } 

X  whatprocessfielding{0-1} 

whatprocessCM{0-1} 

X  whatprocesstoolsuppt{0-1} 

X  whatprocessSWEngSuppt{0-1} 

whatprocessnone{0-1 } 
whatproductscustom{0-1 } 

X  whatproductsCOTS{0-1} 

X  whatproductscommoncust{0-1} 

whatproductsnone{0-1} 


Step  History 

Step  Parameter 

1  whattypecomponentclass{0-1} 

2  whatproductscommoncust{0-1} 

3  whattypesystemscommunications{0-1 } 

4  whattypesystemsembedded{0-1} 

5  whatprocessdesign{0-1} 

6  whatprocesstoolsuppt{0-1} 

7  whatprocessfielding{0-1} 

8  whattypeenterpriseacctng{0-1} 

9  whattypeenterprisescripting{0-1 } 

10  whatprocessmaintenance{0-1} 

11  whatprocessSWEngSuppt{0-1} 

12  whattypeshrinkutilities{0-1} 

1 3  whattypesystemscommunications{0-1 } 

14  whatprocessreengineering{0-1} 

15  whattypesystemsembedded{0-1} 

16  whatproductsCOTS{0-1} 

17  whatprocessrequirements{0-1} 

18  whattypeshrinkbusiness{0-1} 

19  whatprocesstesting{0-1} 

20  whatprocessdesign{0-1} 


Estimate  nDF 

3.1170444  1 

0  1 

0  1 

0  1 

0  1 

0.22845859  1 

-1.1537391  1 

0  1 

0  1 

0  1 

1 .37944707  1 

0  1 

0  1 

0.64617145  1 

0  1 

0  1 

1.08913498  1 

-0.3250263  1 

0  1 

-0.2974418  1 

-0.2170486  1 

-0.5392415  1 

0  1 

0  1 

0  1 

0  1 

-1 .0705623  1 

0  1 

0.60171302  1 

0.79628808  1 

0  1 

0  1 

0.37749312  1 

-1.0747237  1 

0  1 


SS 

0 

0.960316 

0.054908 

0.171437 

0.141001 

1.810994 

10.19188 

0.265902 

0.343736 

0.023415 

16.20785 

0.060774 

0.439882 

4.989225 

0.194504 

0.12867 

7.562117 

2.275132 

0.000717 

2.568616 

1.539893 

5.418224 

0.238358 

0.941234 

0.348658 

0.36637 

7.4685 

0.021905 

4.465961 

9.425311 

0.07275 

0.000006 

3.021779 

19.06236 

0.037064 


"F  Ratio" 
0.000 
1.291 
0.071 
0.222 
0.183 
2.411 
13.567 
0.346 
0.449 
0.030 
21.576 
0.078 
0.577 
6.642 
0.252 
0.167 
10.067 
3.029 
0.001 
3.419 
2.050 
7.213 
0.310 
1.264 
0.456 
0.479 
9.942 
0.028 
5.945 
12.547 
0.094 
0.000 
4.023 
25.376 
0.048 


'Prob>F" 

1.0000 

0.2652 

0.7920 

0.6408 

0.6723 

0.1310 

0.0009 

0.5608 

0.5080 

0.8633 

0.0001 

0.7814 

0.4535 

0.0151 

0.6191 

0.6862 

0.0035 

0.0921 

0.9760 

0.0743 

0.1626 

0.0117 

0.5820 

0.2701 

0.5050 

0.4943 

0.0037 

0.8678 

0.0209 

0.0013 

0.7614 

0.9979 

0.0540 

0.0000 

0.8285 


Action 

Entered 

Entered 

Entered 

Entered 

Entered 

Entered 

Entered 

Entered 

Entered 

Entered 

Entered 

Entered 

Removed 

Entered 

Removed 

Entered 

Entered 

Entered 

Entered 

Removed 


"Sig  Prob"  Seq  SS 

0.0055  15.24444 

0.0056  12.92896 

0.0456  5.987957 

0.0297  6.510128 

0.1212  3.098888 

0.1294  2.863368 

0.1732  2.243077 

0.1100  2.986383 

0.1008  3.007251 

0.1136  2.666641 

0.1662  1.968806 

0.1640  1.931969 

0.2858  1.122046 

0.0922  2.71893 

0.3106  0.957959 

0.0671  3.010756 

0.1868  1.489446 

0.1898  1.433663 

0.1954  1.364919 

0.9760  0.000717 


RSquare  Cp  p 

0.1659  5.4313  2 

0.3065  -0.399  3 

0.3717  -2.025  4 

0.4425  -3.968  5 

0.4762  -3.845  6 

0.5074  -3.579  7 

0.5318  -2.937  8 

0.5643  -2.746  9 

0.5970  -2.567  10 

0.6260  -2.182  11 

0.6474  -1.375  12 

0.6684  -0.545  13 

0.6562  -1.865  12 

0.6858  -1.512  13 

0.6754  -2.932  12 

0.7082  -2.755  13 

0.7244  -1.657  14 

0.7400  -0.525  15 

0.7548  0.6481  16 

0.7548  -1.351  15 


Appendix  B 


Response  Column  59 
Actual  by  Predicted  Plot 


Column  59  Predicted  P<.0001  RSq— 0.75 
RMSE=0.8667 


Summary  of  Fit 


RSquare 
RSquare  Adj 
Root  Mean  Square  Error 
Mean  of  Response 
Observations  (or  Sum  Wgts) 
Analysis  of  Variance 
Source  DF 

Model  14 

Error  30 

C.  Total  44 

Lack  Of  Fit 

Source  DF 

Lack  Of  Fit  22 

Pure  Error  8 

Total  Error  30 


Parameter  Estimates 

Term 


0.754804 

0.640379 

0.866723 

3.844444 

45 


Sum  of  Squares 
69.374866 
22.536245 
91.911111 


Mean  Square  F  Ratio 

4.95535  6.5965 

0.75121  Prob  >  F 

<.0001 


Sum  of  Squares 

18.536245 
4.000000 

22.536245 


Mean  Square  F  Ratio 

0.842557  1.6851 

0.500000  Prob  >  F 

0.2271 
Max  RSq 
0.9565 


Estimate  Std  Error  t  Ratio 


Intercept 

whattypeshrinkbusiness[1  -0] 
whattypeshrinkutilities[1  -0] 
whattypecomponentclass[1  -0] 
whattypeenterpriseacctngjl  -0] 
whattypeenterprisescripting[1  -0] 
whatprocessrequirements[1-0] 
whatprocesstesting[1  -0] 
whatprocessmaintenance[1  -0] 
whatprocessreengineering[1  -0] 
whatprocessfieldingjl  -0] 
whatprocesstoolsuppt[1  -0] 
whatprocessSWEngSuppt[1  -0] 
whatproductsCOTS[1-0] 
whatproductscommoncust[1-0] 
Effect  T ests 


3.5579673 
-0.456917 
2.3074782 
-2.758894 
-1.292343 
-2.17827 
0.6500525 
0.5948836 
0.4340973 
1 .078483 
2.1411246 
-1 .203426 
-1 .592576 
-0.754986 
2.1494475 


0.285331 

0.294279 

0.626456 

0.593954 

0.501466 

0.686547 

0.37353 

0.321708 

0.303195 

0.401573 

0.679056 

0.493562 

0.449606 

0.376433 

0.426696 


12.47 

-1.55 

3.68 
-4.64 
-2.58 
-3.17 

1.74 

1.85 

1.43 

2.69 
3.15 
-2.44 
-3.54 
-2.01 
5.04 


Prob>|t| 

<.0001 

0.1310 

0.0009 

<.0001 

0.0151 

0.0035 

0.0921 

0.0743 

0.1626 

0.0117 

0.0037 

0.0209 

0.0013 

0.0540 

<.0001 


Source 

whattypeshrinkbusiness 

whattypeshrinkutilities 

whattypecomponentclass 

whattypeenterpriseacctng 

whattypeenterprisescripting 

whatprocessrequirements 

whatprocesstesting 

whatprocessmaintenance 

whatprocessreengineering 

whatprocessfielding 

whatprocesstoolsuppt 

whatprocessSWEngSuppt 

whatproductsCOTS 

whatproductscommoncust 


Nparm  DF 

1  1 

1  1 

1  1 

1  1 

1  1 

1  1 

1  1 

1  1 

1  1 

1  1 

1  1 

1  1 

1  1 

1  1 


Sum  of  Squares 
1.810994 
10.191883 
16.207850 
4.989225 
7.562117 
2.275132 
2.568616 
1.539893 
5.418224 
7.468500 
4.465961 
9.42531 1 
3.021779 
19.062365 


F  Ratio  Prob  >  F 

2.4108  0.1310 

13.5673  0.0009 

21.5757  <.0001 

6.6416  0.0151 

10.0666  0.0035 

3.0286  0.0921 

3.4193  0.0743 

2.0499  0.1626 

7.2127  0.0117 

9.9420  0.0037 

5.9450  0.0209 

12.5469  0.0013 

4.0226  0.0540 

25.3756  <.0001 


FailLikely  =  3.56  +  shrink-bus(-0.46)  +  shrink-util(2.31)  +  comp-class(-2.76)  +  ent-acct(-1 .29)  +  ent-script(-2.18)  +  proc- 
req(0.65)  +  proc-test(0.59)  +  proc-maint(0.43)  +  proc-reeng(1 .08)  +  proc-field(2.14)  +  proc-toolsup(-1.20)  +  proc- 
SWEngSup(-1.59)  +  prod-COTS(-0.75)  +  prod-comcust(2.15) 
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Stepwise  Fit  -  New  Survey  Data  -  Consequences  (RespCust) 

Response: 

Column  59 


Stepwise  Regression  Control 

Prob  to  Enter  0.250 

Prob  to  Leave  0.250 

Direction: 

Rules: 


Current  Estimates 

SSE  DFE 

12.611507  27 


MSE  RSquare 

0.4670928  0.7992 


RSquare  Adj  Cp  AIC 

0.6727  5.1267631  -21.2424 


Lock  Entered  Parameter 

X  X  Intercept 

whattypesystemsavionics{0-1 } 
whattypesystemsembedded{0-1 } 
whattypesystemscommunications{0-1} 
whattypesystemsdevice{0- 1 } 

X  whattypeshrinkbusiness{0-1} 

whattypeshrinkutilities{0-1} 

X  whattypeshrinkinternet{0-1} 

X  whattypecomponentdomain{0-1} 

whattypecomponentCASE{0-1} 

X  whattypecomponentclass{0-1} 

whattypecomponentOS{0-1 } 
whattypecomponentdevelopment{0-1} 
X  whattypeenterpriseacctng{0-1} 

whattypeenterprisemanufact{0-1 } 
whattypeenterprisepayroll{0-1 } 

X  whattypeenterprisescripting{0-1} 

X  whatprocessrequirements{0-1} 

X  whatprocessdesign{0-1} 
whatprocesstesting{0- 1 } 

X  whatprocessmaintenance{0-1} 

X  whatprocessreengineering{0-1} 

X  whatprocessappsuppt{0-1} 

X  whatprocesstraining{0-1} 

whatprocessspecification{0-1} 
whatprocesscoding{0- 1 } 
whatprocessfielding{0-1} 
whatprocessCM{0-1 } 
whatprocesstoolsuppt{0-1} 

X  whatprocessSWEngSuppt{0-1} 

X  whatprocessnone{0-1} 

X  whatproductscustom{0-1} 

whatproductsCOTS{0-1} 

X  whatproductscommoncust{0-1} 

X  whatproductsnone{0-1} 

Step  History 

Step  Parameter 

1  whatprocessSWEngSuppt{0-1} 

2  whattypecomponentclass{0-1} 

3  whattypeenterpriseacctng{0-1} 

4  whattypeshrinkinternet{0-1} 

5  whatprocessappsuppt{0-1} 

6  whattypecomponentdomain{0-1} 

7  whattypeshrinkbusiness{0-1} 

8  whatproductscustom{0-1} 

9  whatproductsnone{0-1} 

10  whatprocessnone{0-1} 

1 1  whatprocesstraining{0-1 } 

12  whatprocessmaintenance{0-1} 

13  whatproductscommoncust{0-1} 

14  whatprocessrequirements{0-1} 

15  whatprocessdesign{0-1} 

16  whatprocessreengineering{0-1} 

17  whattypeenterprisescripting{0-1} 


Action 

Entered 

Entered 

Entered 

Entered 

Entered 

Entered 

Entered 

Entered 

Entered 

Entered 

Entered 

Entered 

Entered 

Entered 

Entered 

Entered 

Entered 


Estimate  nDF 

5.08651969  1 

0  1 

0  1 

0  1 

0  1 

0.39217631  1 

0  1 

-0.2604858  1 

-0.6828767  1 

0  1 

1.0254731  1 

0  1 

0  1 

-1.0407701  1 

0  1 

0  1 

0.36852393  1 

-0.43038  1 

0.29930668  1 

0  1 

0.25526518  1 

0.23709166  1 

0.48597345  1 

-0.449339  1 

0  1 

0  1 

0  1 

0  1 

0  1 

-0.4809166  1 

1.19203992  1 

-0.4367591  1 

0  1 

-0.3363608  1 

-1.0517744  1 


"Sig  Prob" 
0.0179 
0.0198 
0.0050 
0.0308 
0.0798 
0.1129 
0.1403 
0.2157 
0.1512 
0.0599 
0.1097 
0.0694 
0.1178 
0.1134 
0.1148 
0.0770 
0.1855 


Seq  SS 
7.754327 
6.749377 
8.527066 
4.428321 
2.707219 
2.114896 
1.76518 
1.214735 
1.596394 
2.601857 
1.766843 
2.143842 
1.497459 
1 .460493 
1.376176 
1.621947 
0.862362 


SS 

0 

0.016113 

0.001794 

0.091147 

0.022034 

4.416902 

0.092277 

1.268235 

3.53688 

0.037501 

7.242644 

0.00563 

0.014271 

10.61653 

0.073591 

0.016593 

0.862362 

3.023544 

2.106273 

0.167372 

1.782155 

1.248765 

4.347448 

4.392837 

0.117749 

0.186928 

0.005728 

0.526168 

0.011487 

4.857045 

5.598886 

3.225526 

0.243773 

1.981869 

7.873346 


RSquare 

0.1235 

0.2310 

0.3667 

0.4372 

0.4804 

0.5140 

0.5421 

0.5615 

0.5869 

0.6283 

0.6565 

0.6906 

0.7145 

0.7377 

0.7596 

0.7854 

0.7992 


"F  Ratio" 
0.000 
0.033 
0.004 
0.189 
0.046 
9.456 
0.192 
2.715 
7.572 
0.078 
15.506 
0.012 
0.029 
22.729 
0.153 
0.034 
1.846 
6.473 
4.509 
0.350 
3.815 
2.673 
9.307 
9.405 
0.245 
0.391 
0.012 
1.132 
0.024 
10.398 
11.987 
6.906 
0.512 
4.243 
16.856 


Cp 

20.659 

15.099 

7.5475 

4.5871 

3.5546 

3.1856 

3.2083 

3.8476 

4.0594 

3.145 

3.1658 

2.7644 

3.087 

3.4511 

3.9096 

4.0927 

5.1268 


'Prob>F" 

1.0000 

0.8567 

0.9520 

0.6671 

0.8327 

0.0048 

0.6652 

0.1110 

0.0105 

0.7829 

0.0005 

0.9150 

0.8651 

0.0001 

0.6992 

0.8546 

0.1855 

0.0170 

0.0430 

0.5594 

0.0612 

0.1136 

0.0051 

0.0049 

0.6247 

0.5371 

0.9143 

0.2971 

0.8788 

0.0033 

0.0018 

0.0140 

0.4805 

0.0492 

0.0003 


P 

2 

3 

4 

5 

6 

7 

8 
9 

10 

11 

12 

13 

14 

15 

16 

17 

18 
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Response  Column  59 
Actual  by  Predicted  Plot 


— i - 1 — i — i - 1 — 

1  2  3  4  5  6  7 

Column  59  Predicted  P<.0001  RSq=0.80 
RMSE=0.6834 


Summary  of  Fit 


RSquare 
RSquare  Adj 
Root  Mean  Square  Error 
Mean  of  Response 
Observations  (or  Sum  Wgts) 
Analysis  of  Variance 


Source  DF 

Model  17 

Error  27 

C.  Total  44 


Parameter  Estimates 

Term 

Intercept 

whattypeshrinkbusiness[1  -0] 
whattypeshrinkinternet[1  -0] 
whattypecomponentdomain[1  -0] 
whattypecomponentclass[1  -0] 
whattypeenterpriseacctng[1  -0] 
whattypeenterprisescripting[1-0] 
whatprocessrequirements[1-0] 
whatprocessdesign[1  -0] 
whatprocessmaintenance[1  -0] 
whatprocessreengineering[1  -0] 
whatprocessappsuppt[1  -0] 
whatprocesstraining[1  -0] 
whatprocessSWEngSuppt[1-0] 
whatprocessnone[1  -0] 
whatproductscustom[1-0] 
whatproductscommoncust[1-0] 
whatproductsnone[1-0] 

Effect  Tests 


0.79918 

0.672737 

0.683442 

4.733333 

45 


Sum  of  Squares 
50.188493 
12.611507 
62.800000 


Mean  Square 
2.95226 
0.46709 


Estimate 

4.1727073 

-0.784353 

0.5209717 

1.3657534 

-2.050946 

2.0815401 

-0.737048 

0.8607599 

-0.598613 

-0.51053 

-0.474183 

-0.971947 

0.8986779 

0.9618333 

-2.38408 

0.8735183 

0.6727217 

2.1035489 


F  Ratio 
6.3205 
Prob  >  F 
<.0001 


Std  Error 
0.310338 
0.255067 
0.316166 
0.496322 
0.520844 
0.436611 
0.542441 
0.338318 
0.281897 
0.261367 
0.290006 
0.318586 
0.293044 
0.298274 
0.688607 
0.332409 
0.326588 
0.512359 


t  Ratio 
13.45 
-3.08 
1.65 
2.75 
-3.94 
4.77 
-1.36 
2.54 
-2.12 
-1.95 
-1.64 
-3.05 
3.07 
3.22 
-3.46 
2.63 
2.06 
4.11 


Prob>|t| 

<.0001 

0.0048 

0.1110 

0.0105 

0.0005 

<.0001 

0.1855 

0.0170 

0.0430 

0.0612 

0.1136 

0.0051 

0.0049 

0.0033 

0.0018 

0.0140 

0.0492 

0.0003 


Source 

whattypeshrinkbusiness 

whattypeshrinkinternet 

whattypecomponentdomain 

whattypecomponentclass 

whattypeenterpriseacctng 

whattypeenterprisescripting 

whatprocessrequirements 

whatprocessdesign 

whatprocessmaintenance 

whatprocessreengineering 

whatprocessappsuppt 

whatprocesstraining 

whatprocessSWEngSuppt 

whatprocessnone 

whatproductscustom 

whatproductscommoncust 

whatprod  uctsnone 


Nparm  DF 

1  1 

1  1 

1  1 

1  1 

1  1 

1  1 

1  1 

1  1 

1  1 

1  1 

1  1 

1  1 

1  1 

1  1 

1  1 

1  1 

1  1 


Sum  of  Squares 
4.416902 
1.268235 
3.536880 
7.242644 
10.616532 
0.862362 
3.023544 
2.106273 
1.782155 
1.248765 
4.347448 
4.392837 
4.857045 
5.598886 
3.225526 
1.981869 
7.873346 


F  Ratio  Prob  >  F 

9.4562  0.0048 

2.7152  0.1110 

7.5721  0.0105 

15.5058  0.0005 

22.7290  <.0001 

1.8462  0.1855 

6.4731  0.0170 

4.5093  0.0430 

3.8154  0.0612 

2.6735  0.1136 

9.3075  0.0051 

9.4046  0.0049 

10.3985  0.0033 

11.9867  0.0018 

6.9055  0.0140 

4.2430  0.0492 

16.8561  0.0003 


ResponseCustomer  =  4.17  +  shrink-bus(-0.78)  +  shrink-int(0.52)  +  comp-domain(1.37)  +  comp-class(-2.05)  +  ent- 
acct(2.08)  +  ent-script(-0.74)  +  proc-req(0.86)  +  proc-des(-0.60)  +  proc-maint(-0.51)  +  proc-reeng(-0.47)  +  procappsup(- 
0.97)  +  proc-train(0.90)  +  proc-SWEngSup(0.96)  +  proc-none(-2.38)  +  prod-cust(0.87)  +  prod-comcust(0.67)  +  prod- 
none(2.10) 
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Stepwise  Fit  -  New  Survey  Data  -  Consequences  (ResponseOrg) 

Response: 

Column  59 


Stepwise  Regression  Control 

Prob  to  Enter  0.250 

Prob  to  Leave  0.250 

Direction: 

Rules: 


Current  Estimates 

SSE  DFE 

32.235426  34 


MSE  RSquare 

0.9481008  0.5583 


RSquare  Adj 
0.4414 


Cp  AIC 

-2.784289  6.310562 


Lock  Entered  Parameter 

X  X  Intercept 

whattypesystemsavionics{0-1 } 
whattypesystemsembedded{0-1 } 

X  whattypesystemscommunications{0-1 } 

whattypesystemsdevice{0- 1 } 
whattypeshrinkbusiness{0-1} 

X  whattypeshrinkutilities{0-1} 

whattypeshrinkinternet{0-1 } 
whattypecomponentdomain{0-1} 
whattypecomponentCASE{0-1} 

X  whattypecomponentclass{0-1} 

whattypecomponentOS{0-1 } 
whattypecomponentdevelopment{0-1} 
X  whattypeenterpriseacctng{0-1} 

whattypeenterprisemanufact{0-1 } 
whattypeenterprisepayroll{0-1 } 
whattypeenterprisescripting{0-1} 

X  whatprocessrequirements{0-1} 

whatprocessdesign{0- 1 } 
whatprocesstesting{0- 1 } 
whatprocessmaintenance{0-1} 
whatprocessreengineering{0-1} 
whatprocessappsuppt{0-1 } 

X  whatprocesstraining{0-1} 

whatprocessspecification{0-1 } 
whatprocesscoding{0- 1 } 

X  whatprocessfielding{0-1} 

whatprocessCM{0-1} 
whatprocesstoolsuppt{0-1} 

X  whatprocessSWEngSuppt{0-1} 

whatprocessnone{0-1 } 
whatproductscustom{0-1 } 
whatproductsCOTS{0-1} 
whatproductscommoncust{0-1} 

X  whatproductsnone{0-1} 

Step  History 

Step  Parameter 

1  whatprocessSWEngSuppt{0-1} 

2  whatprocessrequirements{0-1} 

3  whattypesystemsembedded{0-1} 

4  whattypecomponentclass{0-1} 

5  whattypeenterpriseacctng{0-1} 

6  whatprocessfielding{0-1} 

7  whattypeshrinkutilities{0-1} 

8  whattypesystemsembedded{0-1} 

9  whattypesystemscommunications{0-1 } 

10  whatprocessappsuppt{0-1} 

1 1  whatprocesstraining{0-1 } 

12  whatproductsnone{0-1} 

13  whatprocessappsuppt{0-1} 


Estimate 

4.09677906 

0 

0 

-0.4425837 

0 

0 

0.84069034 


-0.5096238 

0 

0 

0 

0.57268938 

0 


0.76534892 

0 

0 

-1 .2563025 
0 
0 
0 
0 

-0.4398698 


nDF 

1 

1 

1 

1 

1 

1 

1 

1 

1 

1 

1 

1 

1 

1 

1 

1 

1 

1 

1 

1 

1 

1 

1 

1 

1 

1 

1 

1 

1 

1 

1 

1 

1 

1 

1 


SS 

0 

1.236373 

0.001524 

5.680939 

0.239672 

0.371016 

6.396692 

0.007102 

1.015121 

0.04247 

3.396528 

0.219886 

0.006482 

2.955505 

0.093486 

0.019466 

0.060385 

8.446155 

0.883425 

0.022992 

0.068038 

0.031853 

0.709158 

2.729817 

0.043499 

1.107305 

5.501793 

0.511472 

0.00073 

29.65929 

0.323764 

0.024324 

1.01834 

0.063211 

2.738963 


"F  Ratio" 
0.000 
1.316 
0.002 
5.992 
0.247 
0.384 
6.747 
0.007 
1.073 
0.044 
3.582 
0.227 
0.007 
3.117 
0.096 
0.020 
0.062 
8.908 
0.930 
0.024 
0.070 
0.033 
0.742 
2.879 
0.045 
1.174 
5.803 
0.532 
0.001 
31.283 
0.335 
0.025 
1.077 
0.065 
2.889 


'Prob>F" 

1.0000 

0.2595 

0.9687 

0.0197 

0.6224 

0.5396 

0.0138 

0.9326 

0.3078 

0.8360 

0.0669 

0.6372 

0.9356 

0.0864 

0.7587 

0.8886 

0.8050 

0.0052 

0.3419 

0.8790 

0.7933 

0.8577 

0.3951 

0.0989 

0.8341 

0.2865 

0.0216 

0.4709 

0.9784 

0.0000 

0.5668 

0.8755 

0.3070 

0.8006 

0.0983 


Action 

Entered 

Entered 

Entered 

Entered 

Entered 

Entered 

Entered 

Removed 

Entered 

Entered 

Entered 

Entered 

Removed 


"Sig  Prob"  Seq  SS 

0.0033  13.7068 

0.0484  5.433224 

0.0961  3.64543 

0.1714  2.380385 

0.1209  2.970161 

0.1578  2.384912 

0.1578  2.320327 

0.3242  1.114015 

0.0450  4.415578 

0.1762  1.901692 

0.2294  1.47411 

0.1643  1.932399 

0.3951  0.709158 


RSquare  Cp  p 

0.1878  -0.991  2 

0.2623  -2.567  3 

0.3122  -2.966  4 

0.3448  -2.533  5 

0.3855  -2.488  6 

0.4182  -2.057  7 

0.4500  -1.585  8 

0.4348  -2.851  7 

0.4953  -3.757  8 

0.5213  -3.009  9 

0.5415  -1.979  10 

0.5680  -1.251  11 

0.5583  -2.784  10 
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Response  Column  59 


RSquare  0.558281 

RSquare  Adj  0.441356 

Root  Mean  Square  Error  0.973705 

Mean  of  Response  4.522727 

Observations  (or  Sum  Wgts)  44 

Analysis  of  Variance 

Source  DF  Sum  of  Squares  Mean  Square 


Model  9 

Error  34 

C.  Total  43 

Lack  Of  Fit 

Source  DF 

Lack  Of  Fit  12 

Pure  Error  22 

Total  Error  34 


Parameter  Estimates 

Term 

Intercept 

whattypesystemscommunications[1  -( 
whattypeshrinkutilities[1  -0] 
whattypecomponentclass[1  -0] 
whattypeenterpriseacctng[1  -0] 
whatprocessrequirements[1-0] 
whatprocesstraining[1  -0] 
whatprocessfielding[1  -0] 
whatprocessSWEngSuppt[1  -0] 
whatproductsnone[1-0] 

Effect  Tests 

Source 

whattypesystemscommunications 

whattypeshrinkutilities 

whattypecomponentclass 

whattypeenterpriseacctng 

whatprocessrequirements 

whatprocesstraining 

whatprocessfielding 

whatprocessSWEngSuppt 

whatprod  uctsnone 


40.741846  4.52687 

32.235426  0.94810 

72.977273 


Sum  of  Squares  Mean  Square 

20.243760  1.68698 

11.991667  0.54508 

32.235426 


Estimate 

3.9238789 

0.8851673 

-1.681381 

-1.215841 

1.0192476 

-1.145379 

0.6223395 

-1.530698 

2.512605 

0.8797397 


Nparm  DF 

1  1 

1  1 

1  1 

1  1 

1  1 

1  1 

1  1 

1  1 

1  1 


F  Ratio 
4.7747 
Prob  >  F 
0.0004 


F  Ratio 
3.0949 
Prob  >  F 
0.0105 
Max  RSq 
0.8357 


Prob>|t| 

<.0001 

0.0197 

0.0138 

0.0669 

0.0864 

0.0052 

0.0989 

0.0216 

<.0001 

0.0983 


Std  Error  t  Ratio 
0.245094  16.01 

0.361612  2.45 

0.647315  -2.60 

0.642372  -1.89 

0.577286  1.77 

0.383749  -2.98 

0.366765  1.70 

0.635425  -2.41 

0.449232  5.59 

0.517593  1.70 


Sum  of  Squares 
5.680939 
6.396692 
3.396528 
2.955505 
8.446155 
2.729817 
5.501793 
29.659292 
2.738963 


F  Ratio  Prob  >  F 

5.9919  0.0197 

6.7468  0.0138 

3.5825  0.0669 

3.1173  0.0864 

8.9085  0.0052 

2.8792  0.0989 

5.8030  0.0216 

31.2828  <.0001 

2.8889  0.0983 


ResponseOrg  =  3.92  +  sys-comm(0.89)  +  shrink-util(-1 .68)  +  comp-class(-1.22)  +  ent-acct(1.02)  +  proc-req(-1.15)  +  proc- 
train(0.62)  +  proc-field(-1 .53)  +  proc-SWEngSup(2.51)  +  prod-none(0.88) 
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Appendix  C  -  Old  Survey  Data  Consequence  Models 

Stepwise  Fit  -  Old  Survey  Data  -  Consequences  (Cost) 

Response: 

Column  59 

Stepwise  Regression  Control 

Prob  to  Enter  0.250 

Prob  to  Leave  0.250 

Direction: 

Rules: 


Current  Estimates 

SSE  DFE 

94.969149  69 


MSE  RSquare 

1.3763645  0.4358 


RSquare  Adj 
0.3458 


Cp 

0.4149569 


AIC 

36.88734 


Lock 

X 


X  Intercept 

whattypesystemsavionics{0-1 } 

X  whattypesystemsembedded{0-1} 

X  whattypesystemscommunications{0-1} 

whattypesystemsdevice{0-1} 
whattypeshrinkbusiness{0-1} 
whattypeshrinkutilities{0-1 } 
whattypeshrinkinternet{0-1 } 
whattypecomponentdomain{0-1} 

X  whattypecomponentCASE{0-1} 

whattypecomponentclass{0- 1 } 

X  whattypecomponentOS{0-1} 

whattypecomponentdevelopment{0-1} 
X  whattypeenterpriseacctng{0-1} 

X  whattypeenterprisemanufact{0-1} 

whattypeenterprisepayroll{0-1 } 

X  whattypeenterpriseOES{0-1} 

whattypeenterprisescripting{0-1} 
whattypeenterpriseweb{0-1 } 
whatprocessrequirements{0-1} 
whatprocessdesign{0- 1 } 
whatprocesstesting{0- 1 } 
whatprocessmaintenance{0-1 } 
whatprocessreengineering{0-1} 
whatprocessappsuppt{0-1 } 
whatprocesstraining{0-1} 

X  whatprocessspecification{0-1} 

whatprocessdocumentation{0-1 } 
whatprocesscoding{0- 1 } 
whatprocessfielding{0-1} 

X  whatprocessCM{0-1} 

whatprocesstoolsuppt{0-1} 
whatprocessSWEngSuppt{0- 1 } 

X  whatproductscustom{0-1} 

X  whatproductsCOTS{0-1} 

whatproductscommoncust{0-1} 
whatproductsnone{0-1 } 


Step  History 

Step  Parameter 

1  whattypecomponentCASE{0-1} 

2  whatproductscustom{0-1} 

3  whattypeenterprisemanufact{0-1} 

4  whattypecomponentOS{0-1} 

5  whatproductsCOTS{0-1} 

6  whattypeenterpriseacctng{0-1} 

7  whattypeenterpriseOES{0-1} 

8  whatprocessspecification{0-1} 

9  whatprocessCM{0-1} 

1 0  whattypesystemscommunications{0-1 } 

11  whattypesystemsembedded{0-1} 


Estimate  nDF 

5.09503937  1 

0  1 

0.25102501  1 

0.23248605  1 

0  1 

0  1 

0  1 

0  1 

0  1 

1 .30855546  1 

0  1 

-0.6791525  1 

0  1 

-0.4615635  1 

-2.1425555  1 

0  1 

0.76310751  1 

0  1 

0  1 

0  1 

0  1 

0  1 

0  1 

0  1 

0  1 

0  1 

-0.3654913  1 

0  1 

0  1 

0  1 

0.42654828  1 

0  1 

0  1 

-0.5577229  1 

-0.3022904  1 

0  1 

0  1 


SS 

0 

0.193625 
2.134879 
2.345615 
1.052806 
0.195847 
0.978239 
0.557187 
1.382287 
18.08818 
0.30438 
7.662275 
0.157871 
3.434004 
16.09787 
0.30438 
4.005354 
0.471245 
0.115201 
0.289193 
0.935394 
0.364407 
1.697659 
0.09262 
1 .38844 
1.210716 
6.0284 
0.529411 
0.00051 
0.085342 
7.57875 
0.263119 
0.611409 
16.44489 
3.508213 
7.2e-7 
0.02895 


"F  Ratio"  "Prob>F" 

0.000  1.0000 

0.139  0.7105 

1.551  0.2172 

1.704  0.1961 

0.762  0.3857 

0.141  0.7089 

0.708  0.4031 

0.401  0.5285 

1.004  0.3198 

13.142  0.0005 

0.219  0.6416 

5.567  0.0211 

0.113  0.7375 

2.495  0.1188 

11.696  0.0011 

0.219  0.6416 

2.910  0.0925 

0.339  0.5623 

0.083  0.7747 

0.208  0.6500 

0.676  0.4137 

0.262  0.6105 

1.238  0.2698 

0.066  0.7975 

1.009  0.3187 

0.878  0.3520 

4.380  0.0400 

0.381  0.5390 

0.000  0.9848 

0.061  0.8054 

5.506  0.0218 

0.189  0.6652 

0.441  0.5091 

11.948  0.0009 

2.549  0.1149 

0.000  0.9994 

0.021  0.8859 


Action 

Entered 

Entered 

Entered 

Entered 

Entered 

Entered 

Entered 

Entered 

Entered 

Entered 

Entered 


"Sig  Prob"  Seq  SS 

0.0037  17.1415 

0.0054  14.38782 

0.0124  10.74094 

0.1069  4.266343 

0.0736  5.120923 

0.0976  4.274577 

0.0953  4.231291 

0.1755  2.740924 

0.0405  6.093762 

0.2102  2.218877 

0.2172  2.134879 


RSquare  Cp  p 

0.1018  14.398  2 

0.1873  7.6994  3 

0.251 1  3.2058  4 

0.2765  2.6265  5 

0.3069  1.5306  6 

0.3323  0.9463  7 

0.3574  0.3882  8 

0.3737  0.7312  9 

0.4099  -0.953  10 

0.4231  -0.294  1 1 

0.4358  0.415  12 


Appendix  C 


Response  Column  59 
Actual  by  Predicted  Plot 


I 


1  2  3  4  5  6  7  8 

Column  59  Predicted  P<.0001  RSq=0.44 
RMSE=1 .1732 


Summary  of  Fit 

RSquare  0.435785 

RSquare  Adj  0.345838 

Root  Mean  Square  Error  1.173186 

Mean  of  Response  4.345679 

Observations  (or  Sum  Wgts)  81 

Analysis  of  Variance 

Source  DF  Sum  of  Squares  Mean  Square  F  Ratio 


Model  1 1 

Error  69 

C.  Total  80 

Lack  Of  Fit 

Source  DF 

Lack  Of  Fit  22 

Pure  Error  47 

Total  Error  69 

Parameter  Estimates 

Term 

73.35184  6.66835 

94.96915  1.37636 

168.32099 

Sum  of  Squares  Mean  Square 

39.695340  1.80433 

55.273810  1.17604 

94.969149 

Estimate 

4.8449 

Prob  >  F 
<.0001 

F  Ratio 
1.5342 

Prob  >  F 
0.1087 

Max  RSq 
0.6716 

Std  Error 

t  Ratio  Prob>|t| 

Intercept 

3.5679856 

0.295303 

12.08  <.0001 

whattypesystemsembeddedjl  -0] 

-0.50205 

0.403113 

-1.25  0.2172 

whattypesystemscommunications[1-0] 

-0.464972 

0.356176 

-1.31  0.1961 

whattypecomponentCASE[1  -0] 

-2.617111 

0.721924 

-3.63  0.0005 

whattypecomponentOSjl  -0] 

1.3583049 

0.575685 

2.36  0.0211 

whattypeenterpriseacctng[1  -0] 

0.923127 

0.584424 

1.58  0.1188 

whattypeenterprisemanufact[1  -0] 

4.2851109 

1 .252981 

3.42  0.0011 

whattypeenterpriseOES[1  -0] 

-1.526215 

0.894668 

-1.71  0.0925 

whatprocessspecificationjl  -0] 

0.7309825 

0.349279 

2.09  0.0400 

whatprocessCM[1  -0] 

-0.853097 

0.363552 

-2.35  0.0218 

whatproductscustom[1  -0] 

1.1154459 

0.322701 

3.46  0.0009 

whatproductsCOTSjl  -0] 

0.6045809 

0.378685 

1.60  0.1149 

Effect  Tests 

Source 

Nparm  DF 

Sum  of  Squares 

F  Ratio 

Prob  >  F 

whattypesystemsembedded 

1  1 

2.134879 

1.5511 

0.2172 

whattypesystemscommunications 

1  1 

2.345615 

1.7042 

0.1961 

whattypecomponentCASE 

1  1 

18.088182 

13.1420 

0.0005 

whattypecomponentOS 

1  1 

7.662275 

5.5670 

0.0211 

whattypeenterpriseacctng 

1  1 

3.434004 

2.4950 

0.1188 

whattypeenterprisemanufact 

1  1 

16.097871 

11.6959 

0.0011 

whattypeenterpriseOES 

1  1 

4.005354 

2.9101 

0.0925 

whatprocessspecification 

1  1 

6.028400 

4.3799 

0.0400 

whatprocessCM 

1  1 

7.578750 

5.5064 

0.0218 

whatproductscustom 

1  1 

16.444887 

11.9481 

0.0009 

whatproductsCOTS 

1  1 

3.508213 

2.5489 

0.1149 

Cost  =  3.57  +  (-.50)sys-embed  +  (-.46)sys-comm  +  (-2.62)comp-case  +  (1.36)comp-os  +  (.92)ent-acct  +  (4.29)ent-mnft 
(-1 .53)ent-oes  +  (.73)proc-spec  +  (-.85)proc-cm  +  (1.12)prod-cust  +  (.60)prod-cots 


Stepwise  Fit  -  Old  Survey  Data  -  Consequences  (Sched) 

Response: 

Column  59 


Stepwise  Regression  Control 

Prob  to  Enter  0.250 

Prob  to  Leave  0.250 

Direction: 

Rules: 


Current  Estimates 

SSE  DFE 

132.06128  69 


MSE  RSquare 

1.9139316  0.3376 


RSquare  Adj 
0.2320 


Cp  AIC 

-3.494273  63.59417 


Lock 

X 


X  Intercept 

X  whattypesystemsavionics{0-1} 

whattypesystemsembedded{0-1 } 

X  whattypesystemscommunications{0-1} 

whattypesystemsdevice{0-1} 

X  whattypeshrinkbusiness{0-1} 

whattypeshrinkutilities{0-1 } 
whattypeshrinkinternet{0-1 } 
whattypecomponentdomain{0-1} 
whattypecomponentCASE{0-1 } 
whattypecomponentclass{0- 1 } 
whattypecomponentOS{0-1 } 
whattypecomponentdevelopment{0-1} 
X  whattypeenterpriseacctng{0-1} 

X  whattypeenterprisemanufact{0-1} 

whattypeenterprisepayroll{0-1 } 
whattypeenterpriseOES{0- 1 } 
whattypeenterprisescripting{0-1} 
whattypeenterpriseweb{0-1 } 
whatprocessrequirements{0-1} 

X  whatprocessdesign{0-1} 

whatprocesstesting{0-1} 
whatprocessmaintenance{0-1} 

X  whatprocessreengineering{0-1} 

whatprocessappsuppt{0-1 } 
whatprocesstraining{0-1} 
whatprocessspecification{0-1 } 

X  whatprocessdocumentation{0-1} 

whatprocesscoding{0- 1 } 

X  whatprocessfielding{0-1} 

X  whatprocessCM{0-1} 

whatprocesstoolsuppt{0-1} 
whatprocessSWEngSuppt{0- 1 } 

X  whatproductscustom{0-1} 

whatproductsCOTS{0-1} 
whatproductscommoncust{0-1} 
whatproductsnone{0-1 } 


Step  History 

Step  Parameter 

1  whatproductscustom{0-1} 

2  whattypeshrinkbusiness{0-1} 

3  whattypeenterpriseacctng{0-1} 

4  whattypeenterprisemanufact{0-1} 

5  whatprocessCM{0-1} 

6  whatprocessdesign{0-1} 

7  whattypecomponentdevelopment{0- 1 } 

8  whattypesystemscommunications{0-1 } 

9  whatprocessfielding{0-1} 

10  whattypesystemsavionics{0-1} 

1 1  whatproductscommoncust{0-1 } 

12  whatprocessreengineering{0-1} 

13  whatprocessdocumentation{0-1} 

1 4  whattypecomponentdevelopment{0- 1 } 

15  whatprocesstoolsuppt{0-1} 

16  whatproductscommoncust{0-1} 

17  whatprocesstoolsuppt{0-1} 


Estimate  nDF 

5.76088737  1 

-0.7850392  1 

0  1 

0.34792507  1 

0  1 

-0.3395791  1 

0  1 

0  1 

0  1 

0  1 

0  1 

0  1 

0  1 

0.42626338  1 

-1.3623693  1 

0  1 

0  1 

0  1 

0  1 

0  1 

-0.5159009  1 

0  1 

0  1 

-0.4367876  1 

0  1 

0  1 

0  1 

0.4316149  1 

0  1 

-0.7039892  1 

0.85920309  1 

0  1 

0  1 

-0.2651114  1 

0  1 

0  1 

0  1 


SS 

0 

9.540614 

1.324173 

5.195854 

0.906236 

3.550993 

1.563745 

1.59107 

0.299425 

0.738799 

0.022533 

0.001029 

1.34129 

3.159379 

6.404336 

0.022533 

0.969841 

0.990601 

2.239416 

0.03184 

12.25879 

0.136139 

2.047242 

7.713886 

0.226483 

0.827338 

0.007269 

6.45517 

2.094378 

13.32058 

24.3518 

2.428116 

0.854875 

3.698955 

0.299977 

1.840536 

0.233932 


"F  Ratio" 
0.000 
4.985 
0.689 
2.715 
0.470 
1.855 
0.815 
0.829 
0.155 
0.383 
0.012 
0.001 
0.698 
1.651 
3.346 
0.012 
0.503 
0.514 
1.173 
0.016 
6.405 
0.070 
1.071 
4.030 
0.117 
0.429 
0.004 
3.373 
1.096 
6.960 
12.723 
1.274 
0.443 
1.933 
0.155 
0.961 
0.121 


'Prob>F" 

1.0000 

0.0288 

0.4095 

0.1040 

0.4954 

0.1776 

0.3699 

0.3657 

0.6955 

0.5383 

0.9145 

0.9817 

0.4065 

0.2032 

0.0717 

0.9145 

0.4806 

0.4759 

0.2826 

0.8985 

0.0137 

0.7919 

0.3044 

0.0486 

0.7336 

0.5148 

0.9514 

0.0706 

0.2989 

0.0103 

0.0007 

0.2630 

0.5079 

0.1689 

0.6952 

0.3304 

0.7294 


Action 

Entered 

Entered 

Entered 

Entered 

Entered 

Entered 

Entered 

Entered 

Entered 

Entered 

Entered 

Entered 

Entered 

Removed 

Entered 

Removed 

Removed 


"Sig  Prob"  Seq  SS 

0.0766  7.805393 

0.0798  7.438212 

0.0882  6.864043 

0.1476  4.85441 

0.1513  4.699276 

0.0850  6.634945 

0.0812  6.613531 

0.0971  5.830852 

0.0973  5.684041 

0.1254  4.748008 

0.1976  3.307965 

0.1845  3.473557 

0.2166  2.982357 

0.3361  1.799309 

0.2072  3.078749 

0.2560  2.491168 

0.2630  2.428116 


RSquare  Cp  p 

0.0392  0.6091  2 

0.0765  -0.405  3 

0.1109  -1.186  4 

0.1352  -1.152  5 

0.1588  -1.056  6 

0.1921  -1.744  7 

0.2253  -2.424  8 

0.2545  -2.786  9 

0.2830  -3.089  10 

0.3068  -3.013  11 

0.3234  -2.353  12 

0.3409  -1.761  13 

0.3558  -0.969  14 

0.3468  -2.24  13 

0.3622  -1.487  14 

0.3497  -2.478  13 

0.3376  -3.494  12 


Appendix  C 


Response  Column  59 
Actual  by  Predicted  Plot 

I  8 

7 

I6 

<5 

8  4 
c 

I3 
02 
1 
0 

1  2  3  4  5  6  7  8 

Column  59  Predicted  P=0.0015  RSq=0.34 
RMSE=  1.3834 


Summary  of  Fit 


RSquare 

0.337567 

RSquare  Adj 

0.231962 

Root  Mean  Square  Error 

1.383449 

Mean  of  Response 

4.395062 

Observations  (or  Sum  Wgts) 

Analysis  of  Variance 

81 

Source  DF 

Sum  of  Squares  Mean  Square 

F  Ratio 

Model  1 1 

67.29675 

6.11789 

3.1965 

Error  69 

132.06128 

1.91393 

Prob  >  F 

C.  Total  80 

199.35802 

0.0015 

Lack  Of  Fit 

Source  DF 

Sum  of  Squares 

Mean  Square 

F  Ratio 

Lack  Of  Fit  32 

67.84699 

2.12022 

1.2217 

Pure  Error  37 

64.21429 

1.73552 

Prob  >  F 

Total  Error  69 

132.06128 

0.2771 

Max  RSq 

0.6779 

Parameter  Estimates 

Term 

Estimate 

Std  Error 

t  Ratio  Prob>|t| 

Intercept 

3.417117 

0.38475 

8.88  <.0001 

whattypesystemsavionics[1  -0] 

1.5700784 

0.703228 

2.23  0.0288 

whattypesystemscommunications[1-0] 

-0.69585 

0.422329 

-1.65  0.1040 

whattypeshrinkbusiness[1  -0] 

0.6791582 

0.498608 

1.36  0.1776 

whattypeenterpriseacctng[1  -0] 

-0.852527 

0.663545 

-1.28  0.2032 

whattypeenterprisemanufact[1  -0] 

2.7247386 

1 .489536 

1.83  0.0717 

whatprocessdesign[1  -0] 

1.0318019 

0.407695 

2.53  0.0137 

whatprocessreengineeringjl  -0] 

0.8735753 

0.435138 

2.01  0.0486 

whatprocessdocumentationjl  -0] 

-0.86323 

0.470041 

-1.84  0.0706 

whatprocessfieldingjl  -0] 

1.4079785 

0.533701 

2.64  0.0103 

whatprocessCM[1  -0] 

-1.718406 

0.481752 

-3.57  0.0007 

whatproductscustom[1  -0] 

Effect  Tests 

0.5302227 

0.381401 

1.39  0.1689 

Source 

Nparm 

DF 

Sum  of  Squares 

F  Ratio 

Prob  >  F 

whattypesystemsavionics 

1 

1 

9.540614 

4.9848 

0.0288 

whattypesystemscommunications 

1 

1 

5.195854 

2.7148 

0.1040 

whattypeshrinkbusiness 

1 

1 

3.550993 

1.8553 

0.1776 

whattypeenterpriseacctng 

1 

1 

3.159379 

1.6507 

0.2032 

whattypeenterprisemanufact 

1 

1 

6.404336 

3.3462 

0.0717 

whatprocessdesign 

1 

1 

12.258794 

6.4050 

0.0137 

whatprocessreengineering 

1 

1 

7.713886 

4.0304 

0.0486 

whatprocessdocumentation 

1 

1 

6.455170 

3.3727 

0.0706 

whatprocessfielding 

1 

1 

13.320581 

6.9598 

0.0103 

whatprocessCM 

1 

1 

24.351798 

12.7234 

0.0007 

whatproductscustom 

1 

1 

3.698955 

1 .9326 

0.1689 

Sched  =  3.42  +  (1.57)sys-avia  +  (-.70)sys-comm  +  (-.68)shrink-bus  +  (-.85)ent-acct  +  (2.72)ent-mnft  +  (1.03)proc-des  + 
(.87)proc-reeng  +  (-.86)proc-doc  +  (1.41)proc-field  +  (-1 .72)proc-cm  +  (.53)prod-cust 
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Stepwise  Fit  -  Old  Survey  Data  -  Consequences  (IntelCapital) 

Response: 

Column  59 


Stepwise  Regression  Control 

Prob  to  Enter  0.250 

Prob  to  Leave  0.250 

Direction: 

Rules: 


Current  Estimates 

SSE  DFE 

30.867642  64 


MSE  RSquare 

0.4823069  0.6174 


RSquare  Adj 
0.5337 


Cp  AIC 

0.380703  -44.2394 


Lock 

X 


X  Intercept 

whattypesystemsavionics{0-1} 
whattypesystemsembedded{0-1 } 
whattypesystemscommunications{0-1} 
whattypesystemsdevice{0-1} 

X  whattypeshrinkbusiness{0-1} 

X  whattypeshrinkutilities{0-1} 

whattypeshrinkinternet{0-1 } 

X  whattypecomponentdomain{0-1} 

X  whattypecomponentCASE{0-1} 

whattypecomponentclass{0- 1 } 
whattypecomponentOS{0-1} 

X  whattypecomponentdevelopment{0-1} 

whattypeenterpriseacctng{0-1 } 

X  whattypeenterprisemanufact{0-1} 

whattypeenterprisepayroll{0-1 } 
whattypeenterpriseOES{0- 1 } 

X  whattypeenterprisescripting{0-1} 

whattypeenterpriseweb{0-1 } 
whatprocessrequirements{0-1} 
whatprocessdesign{0- 1 } 
whatprocesstesting{0- 1 } 
whatprocessmaintenance{0-1} 

X  whatprocessreengineering{0-1} 

X  whatprocessappsuppt{0-1} 

X  whatprocesstraining{0-1} 

whatprocessspecification{0-1 } 
whatprocessdocumentation{0-1 } 

X  whatprocesscoding{0-1} 

X  whatprocessfielding{0-1} 

whatprocessCM{0-1} 
whatprocesstoolsuppt{0-1} 

X  whatprocessSWEngSuppt{0-1} 

X  whatproductscustom{0-1} 

whatproductsCOTS{0-1} 
whatproductscommoncust{0-1} 
whatproductsnone{0-1 } 


Step  History 

Step  Parameter 

1  whatprocessfielding{0-1} 

2  whatprocesstraining{0-1} 

3  whattypeshrinkbusiness{0-1} 

4  whattypecomponentCASE{0-1} 

5  whattypecomponentdomain{0-1} 

6  whatproductscustom{0-1} 

7  whatprocesscoding{0-1} 

8  whattypeshrinkutilities{0-1} 

9  whattypecomponentdevelopment{0- 1 } 

10  whatprocessreengineering{0-1} 

11  whatprocessSWEngSuppt{0-1} 

12  whatprocessappsuppt{0-1} 

13  whattypeenterprisemanufact{0-1} 

14  whattypeenterprisescripting{0-1} 


Action 

Entered 

Entered 

Entered 

Entered 

Entered 

Entered 

Entered 

Entered 

Entered 

Entered 

Entered 

Entered 

Entered 

Entered 


Estimate  nDF 

4.85966053  1 

0  1 

0  1 

0  1 

0  1 

-0.5474613  1 

0.49462323  1 

0  1 

-0.6837601  1 

0.82830633  1 

0  1 

0  1 

0.32724112  1 

0  1 

-0.701835  1 

0  1 

0  1 

-0.336725  1 

0  1 

0  1 

0  1 

0  1 

0  1 

-0.2544066  1 

-0.2971724  1 

-0.4768208  1 

0  1 

0  1 

0.22224395  1 

0.83925955  1 

0  1 

0  1 

0.29495325  1 

0.14679207  1 

0  1 

0  1 

0  1 


SS 

0 

0.085086 
0.03452 
0.25992 
0.035419 
8.887617 
1.624533 
0.087749 
7.348932 
6.232985 
0.079747 
0.007463 
4.304229 
0.011283 
1 .562465 
0.079747 
0.214318 
1 .50839 
0.120047 
0.000857 
0.017324 
0.468761 
0.006668 
1.844106 
1.781561 
6.336608 
0.172396 
0.01511 
1.685393 
17.94046 
0.378987 
0.121866 
3.919188 
1.176843 
0.37651 
0.573181 
0.557566 


"F  Ratio" 
0.000 
0.174 
0.071 
0.535 
0.072 
18.427 
3.368 
0.180 
15.237 
12.923 
0.163 
0.015 
8.924 
0.023 
3.240 
0.163 
0.440 
3.127 
0.246 
0.002 
0.035 
0.971 
0.014 
3.824 
3.694 
13.138 
0.354 
0.031 
3.494 
37.197 
0.783 
0.250 
8.126 
2.440 
0.778 
1.192 
1.159 


'Prob>F" 

1.0000 

0.6779 

0.7914 

0.4672 

0.7888 

0.0001 

0.0711 

0.6732 

0.0002 

0.0006 

0.6876 

0.9022 

0.0040 

0.8798 

0.0766 

0.6876 

0.5093 

0.0817 

0.6217 

0.9668 

0.8514 

0.3281 

0.9075 

0.0549 

0.0591 

0.0006 

0.5541 

0.8611 

0.0662 

0.0000 

0.3796 

0.6190 

0.0059 

0.1232 

0.3811 

0.2791 

0.2858 


'Sig  Prob"  Seq  SS 

0.0004  12.08328 

0.0105  5.699798 

0.0258  4.06017 

0.0032  6.565 

0.0346  3.121486 

0.0370  2.902028 

0.0598  2.267616 

0.0835  1.855673 

0.1067  1.570788 

0.0760  1.848725 

0.0463  2.244364 

0.0287  2.569127 

0.0854  1.519463 

0.0817  1.50839 


RSquare  Cp  p 

0.1498  34.744  2 

0.2204  27.625  3 

0.2707  23.13  4 

0.3521  14.628  5 

0.3908  11.634  6 

0.4268  8.9916  7 

0.4549  7.364  8 

0.4779  6.3953  9 

0.4973  5.8824  10 

0.5202  4.9249  1 1 

0.5481  3.3345  12 

0.5799  1.2245  13 

0.5987  0.7938  14 

0.6174  0.3807  15 


Appendix  C 


Response  Column  59 
Actual  by  Predicted  Plot 


1  2  3  4  5  6  7 

Column  59  Predicted  P<.0001  RSq=0.62 
RMSE=0.6945 


Summary  of  Fit 


RSquare 
RSquare  Adj 
Root  Mean  Square  Error 
Mean  of  Response 
Observations  (or  Sum  Wgts) 
Analysis  of  Variance 
Source  DF 

Model  14 

Error  64 

C.  Total  78 

Lack  Of  Fit 

Source  DF 

Lack  Of  Fit  31 

Pure  Error  33 

Total  Error  64 


0.617423 

0.533735 

0.694483 

4.063291 

79 


Sum  of  Squares 
49.815903 
30.867642 
80.683544 

Sum  of  Squares 
26.034308 
4.833333 
30.867642 


Mean  Square 
3.55828 
0.48231 


Mean  Square 
0.839816 
0.146465 


Parameter  Estimates 

Term 

Intercept 

whattypeshrinkbusiness[1  -0] 
whattypeshrinkutilities[1-0] 
whattypecomponentdomain[1  -0] 
whattypecomponentCASE[1  -0] 
whattypecomponentdevelopment[1-0] 
whattypeenterprisemanufact[1  -0] 
whattypeenterprisescripting[1  -0] 
whatprocessreengineeringjl  -0] 
whatprocessappsupptjl  -0] 
whatprocesstraining[1  -0] 
whatprocesscoding[1  -0] 
whatprocessfieldingjl  -0] 
whatprocessSWEngSuppt[1  -0] 
whatproductscustom[1  -0] 

Effect  Tests 

Source 

whattypeshrinkbusiness 

whattypeshrinkutilities 

whattypecomponentdomain 

whattypecomponentCASE 

whattypecomponentdevelopment 

whattypeenterprisemanufact 

whattypeenterprisescripting 

whatprocessreengineering 

whatprocessappsuppt 

whatprocesstraining 

whatprocesscoding 

whatprocessfielding 

whatprocessSWEngSuppt 

whatproductscustom 


Estimate 
4.7148988 
1.0949227 
-0.989246 
1.3675202 
-1.656613 
-0.654482 
1 .40367 
0.67345 
0.5088132 
0.5943447 
0.9536417 
-0.444488 
-1.678519 
-0.589907 
-0.293584 


Nparm  DF 

1  1 

1  1 

1  1 

1  1 

1  1 

1  1 

1  1 

1  1 

1  1 

1  1 

1  1 

1  1 

1  1 

1  1 


F  Ratio 
7.3776 
Prob  >  F 
<.0001 


F  Ratio 
5.7339 
Prob  >  F 
<.0001 
Max  RSq 
0.9401 


Std  Error 
0.24903 
0.255066 
0.539016 
0.350335 
0.460824 
0.219085 
0.779869 
0.380812 
0.260212 
0.309243 
0.263099 
0.237778 
0.275214 
0.206941 
0.187947 


Sum  of  Squares 
8.887617 
1 .624533 
7.348932 
6.232985 
4.304229 
1.562465 
1 .508390 
1.844106 
1.781561 
6.336608 
1 .685393 
17.940456 
3.919188 
1.176843 


t  Ratio 
18.93 
4.29 
-1.84 
3.90 
-3.59 
-2.99 
1.80 
1.77 
1.96 
1.92 
3.62 
-1.87 
-6.10 
-2.85 
-1.56 


Prob>|t| 

<.0001 

<.0001 

0.0711 

0.0002 

0.0006 

0.0040 

0.0766 

0.0817 

0.0549 

0.0591 

0.0006 

0.0662 

<.0001 

0.0059 

0.1232 


Prob  >  F 
<.0001 
0.0711 
0.0002 
0.0006 
0.0040 
0.0766 
0.0817 
0.0549 
0.0591 
0.0006 
0.0662 
<.0001 
0.0059 
0.1232 


F  Ratio 
18.4273 
3.3683 
15.2370 
12.9233 
8.9243 
3.2396 
3.1274 
3.8235 
3.6938 
13.1381 
3.4944 
37.1972 
8.1259 
2.4400 


IntelCap  =  4.71  +  (1.09)shrink-bus  +  (-.99)shrink-util  +  (1.37)comp-domain  +  (-1.67)comp-case  +  (-.65)comp-dev  + 
(1.40)ent-mnft  +  (.67)ent-script  +  (.51)proc-reeng  +  (.59)proc-appsup  +  (.95)proc-train  +  (-.44)proc-coding  +  (-1.68)proc- 
field  +  (-.59)proc-swengsup  +  (-.29)prod-cust 
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Stepwise  Fit  -  Old  Survey  Data  -  Consequences  (SchedFlex) 

Response: 

Column  59 


Stepwise  Regression  Control 

Prob  to  Enter  0.250 

Prob  to  Leave  0.250 

Direction: 

Rules: 


Current  Estimates 

SSE  DFE 

109.55497  70 


MSE  RSquare 

1.5650711  0.3352 


RSquare  Adj 
0.2498 


Cp  AIC 

-7.411039  45.15199 


Lock 

X 


X  Intercept 

X  whattypesystemsavionics{0-1} 

whattypesystemsembedded{0-1 } 

X  whattypesystemscommunications{0-1 } 

whattypesystemsdevice{0-1} 
whattypeshrinkbusiness{0-1} 
whattypeshrinkutilities{0-1 } 
whattypeshrinkinternet{0-1 } 
whattypecomponentdomain{0-1} 

X  whattypecomponentCASE{0-1} 

whattypecomponentclass{0- 1 } 

X  whattypecomponentOS{0-1} 

whattypecomponentdevelopment{0-1} 
X  whattypeenterpriseacctng{0-1} 

whattypeenterprisemanufact{0-1 } 
whattypeenterprisepayroll{0-1 } 
whattypeenterpriseOES{0- 1 } 
whattypeenterprisescripting{0-1} 
whattypeenterpriseweb{0-1 } 

X  whatprocessrequirements{0-1} 

whatprocessdesign{0- 1 } 
whatprocesstesting{0- 1 } 
whatprocessmaintenance{0-1} 

X  whatprocessreengineering{0-1} 

whatprocessappsuppt{0-1 } 
whatprocesstraining{0-1} 

X  whatprocessspecification{0-1} 

whatprocessdocumentation{0-1 } 
whatprocesscoding{0- 1 } 

X  whatprocessfielding{0-1} 

whatprocessCM{0-1} 
whatprocesstoolsuppt{0-1} 
whatprocessSWEngSuppt{0- 1 } 
whatproductscustom{0-1 } 
whatproductsCOTS{0-1} 
whatproductscommoncust{0-1} 
whatproductsnone{0-1 } 


Step  History 

Step  Parameter 

1  whattypeenterpriseacctng{0-1} 

2  whattypesystemsavionics{0-1} 

3  whatprocessfielding{0-1} 

4  whattypesystemscommunications{0-1 } 

5  whattypecomponentCASE{0-1} 

6  whatprocessreengineering{0-1} 

7  whatprocessrequirements{0-1} 

8  whatprocessspecification{0-1} 

9  whattypecomponentOS{0-1} 


Estimate  nDF 

3.4152258  1 

0.67585057  1 

0  1 

-0.5311669  1 

0  1 

0  1 

0  1 

0  1 

0  1 

0.76677462  1 

0  1 

0.45179531  1 

0  1 

-0.7581915  1 

0  1 

0  1 

0  1 

0  1 

0  1 

0.57159557  1 

0  1 

0  1 

0  1 

-0.3797237  1 

0  1 

0  1 

-0.4325519  1 

0  1 

0  1 

0.41524402  1 

0  1 

0  1 

0  1 

0  1 

0  1 

0  1 

0  1 


SS 

0 

8.171748 
0.525227 
12.28546 
0.002158 
0.216543 
0.747146 
0.556425 
1 .97971 
6.282651 
0.005051 
3.711625 
0.139061 
9.800447 
0.147458 
0.005051 
1 .248898 
0.034714 
1.214833 
12.89219 
0.047718 
0.082181 
0.004937 
5.612145 
0.621101 
0.993241 
7.589665 
0.041942 
1.935249 
6.795611 
1.548536 
0.127274 
0.770412 
2.062971 
0.402375 
0.213559 
0.19748 


"F  Ratio"  "Prob>F" 

0.000  1.0000 

5.221  0.0253 

0.332  0.5661 

7.850  0.0066 

0.001  0.9707 

0.137  0.7128 

0.474  0.4936 

0.352  0.5548 

1.270  0.2637 

4.014  0.0490 

0.003  0.9552 

2.372  0.1281 

0.088  0.7680 

6.262  0.0147 

0.093  0.7613 

0.003  0.9552 

0.796  0.3755 

0.022  0.8829 

0.774  0.3821 

8.237  0.0054 

0.030  0.8628 

0.052  0.8206 

0.003  0.9557 

3.586  0.0624 

0.393  0.5326 

0.631  0.4296 

4.849  0.0310 

0.026  0.8713 

1.241  0.2692 

4.342  0.0408 

0.989  0.3234 

0.080  0.7778 

0.489  0.4869 

1.324  0.2538 

0.254  0.6156 

0.135  0.7147 

0.125  0.7252 


Action 

Entered 

Entered 

Entered 

Entered 

Entered 

Entered 

Entered 

Entered 

Entered 


"Sig  Prob"  Seq  SS 

0.0242  10.45333 

0.0683  6.560952 

0.0615  6.688278 

0.1273  4.334386 

0.0719  5.896956 

0.1382  3.907936 

0.0582  6.213503 

0.0337  7.478056 

0.1281  3.711625 


RSquare  Cp  p 

0.0634  -1.91  2 

0.1032  -3.059  3 

0.1438  -4.27  4 

0.1701  -4.351  5 

0.2059  -5.181  6 

0.2296  -5.057  7 

0.2673  -6.04  8 

0.3127  -7.629  9 

0.3352  -7.411  10 


Appendix  C 


Response  Column  59 
Actual  by  Predicted  Plot 


I 

8 

l 


1  2  3  4  5  6  7 

Column  59  Predicted  P=0.0004  RSq=0.34 
RMSE=1.251 


Summary  of  Fit 


RSquare  0.335225 

RSquare  Adj  0.249754 

Root  Mean  Square  Error  1.251028 

Mean  of  Response  4.2 

Observations  (or  Sum  Wgts)  80 


Analysis  of  Variance 

Source 

DF 

Sum  of  Squares 

Mean  Square 

Model 

9 

55.24503 

6.13834 

Error 

70 

109.55497 

1 .56507 

C.  Total 

79 

164.80000 

F  Ratio 
3.9221 
Prob  >  F 
0.0004 


Lack  Of  Fit 

Source 
Lack  Of  Fit 
Pure  Error 
Total  Error 


DF 

21 

49 

70 


Sum  of  Squares  Mean  Square 

30.05497  1.43119 

79.50000  1.62245 

109.55497 


F  Ratio 
0.8821 
Prob  >  F 
0.6122 
Max  RSq 
0.5176 


Parameter  Estimates 

Term 

Intercept 

whattypesystemsavionics[1  -0] 
whattypesystemscommunications[1-0] 
whattypecomponentCASE[1  -0] 
whattypecomponentOS[1  -0] 
whattypeenterpriseacctng[1  -0] 
whatprocessrequirementsjl  -0] 
whatprocessreengineering[1  -0] 
whatprocessspecification[1-0] 
whatprocessfielding[1  -0] 


Estimate 

4.1948518 

-1.351701 

1.0623337 

-1.533549 

-0.903591 

1.5163831 

-1.143191 

0.7594474 

0.8651039 

-0.830488 


Std  Error 
0.211127 
0.591548 
0.379168 
0.765409 
0.586755 
0.605973 
0.398311 
0.401052 
0.392847 
0.398553 


t  Ratio 
19.87 
-2.29 
2.80 
-2.00 
-1.54 
2.50 
-2.87 
1.89 
2.20 
-2.08 


Prob>|t| 

<.0001 

0.0253 

0.0066 

0.0490 

0.1281 

0.0147 

0.0054 

0.0624 

0.0310 

0.0408 


Effect  Tests 

Source 

whattypesystemsavionics 

whattypesystemscommunications 

whattypecomponentCASE 

whattypecomponentOS 

whattypeenterpriseacctng 

whatprocessrequirements 

whatprocessreengineering 

whatprocessspecification 

whatprocessfielding 


Nparm  DF 

1  1 

1  1 

1  1 

1  1 

1  1 

1  1 

1  1 

1  1 

1  1 


Sum  of  Squares 
8.171748 
12.285464 
6.282651 
3.711625 
9.800447 
12.892188 
5.612145 
7.589665 
6.795611 


F  Ratio  Prob  >  F 

5.2213  0.0253 

7.8498  0.0066 

4.0143  0.0490 

2.3715  0.1281 

6.2620  0.0147 

8.2374  0.0054 

3.5859  0.0624 

4.8494  0.0310 

4.3420  0.0408 


SchedFlex  =  4.19  +  sys-avia(-1.35)  +  sys-comm(1.06)  +  comp-CASE(-1 .53)  +  comp-OS(-0.90)  +  ent-acct(1.52)  +  proc- 
req(-1.14)  +  proc-reeng(0.76)  +  proc-spec(0.87)  +  proc-field(-0.83) 
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Stepwise  Fit  -  Old  Survey  Data  -  Consequences  (AdminOverhead) 

Response: 

Column  59 

Stepwise  Regression  Control 

Prob  to  Enter  0.250 

Prob  to  Leave  0.250 

Direction: 

Rules: 


Current  Estimates 

SSE  DFE 

74.991596  68 


MSE  RSquare 

1.1028176  0.4314 


RSquare  Adj  Cp  AIC 

0.3394  -5.033458  18.82795 


Lock 

X 


X  Intercept 

X  whattypesystemsavionics{0-1} 

whattypesystemsembedded{0-1} 
whattypesystemscommunications{0-1} 
whattypesystemsdevice{0-1} 
whattypeshrinkbusiness{0-1} 

X  whattypeshrinkutilities{0-1} 

X  whattypeshrinkinternet{0-1} 

X  whattypecomponentdomain{0-1} 

whattypecomponentCASE{0-1} 
whattypecomponentclass{0- 1 } 
whattypecomponentOS{0-1} 
whattypecomponentdevelopment{0-1} 
whattypeenterpriseacctng{0-1 } 

X  whattypeenterprisemanufact{0-1} 

whattypeenterprisepayroll{0-1 } 

X  whattypeenterpriseOES{0-1} 

whattypeenterprisescripting{0-1} 
whattypeenterpriseweb{0-1 } 
whatprocessrequirements{0-1} 
whatprocessdesign{0- 1 } 
whatprocesstesting{0- 1 } 
whatprocessmaintenance{0-1} 
whatprocessreengineering{0-1 } 

X  whatprocessappsuppt{0-1} 

whatprocesstraining{0-1} 
whatprocessspecification{0-1 } 

X  whatprocessdocumentation{0-1} 

whatprocesscoding{0- 1 } 

X  whatprocessfielding{0-1} 

X  whatprocessCM{0-1} 

whatprocesstoolsuppt{0-1} 
whatprocessSWEngSuppt{0- 1 } 
whatproductscustom{0-1 } 
whatproductsCOTS{0-1} 

X  whatproductscommoncust{0-1} 

whatproductsnone{0-1 } 


Step  History 

Step  Parameter 

1  whatprocessdocumentation{0-1} 

2  whattypeenterprisemanufact{0-1} 

3  whattypeshrinkutilities{0-1} 

4  whattypesystemsavionics{0-1} 

5  whattypeshrinkinternet{0-1} 

6  whatprocessappsuppt{0-1} 

7  whatprocessCM{0-1} 

8  whattypecomponentdomain{0-1} 

9  whatproductscommoncust{0-1} 

10  whattypeenterpriseOES{0-1} 

11  whatprocessfielding{0-1} 


Action 

Entered 

Entered 

Entered 

Entered 

Entered 

Entered 

Entered 

Entered 

Entered 

Entered 

Entered 


Estimate  nDF 

7.4199209  1 

-1.1546067  1 

0  1 

0  1 

0  1 

0  1 

-1.3081713  1 

-0.6296412  1 

0.7359386  1 

0  1 

0  1 

0  1 

0  1 

0  1 

-1.4334054  1 

0  1 

0.63003481  1 

0  1 

0  1 

0  1 

0  1 

0  1 

0  1 

0  1 

-0.4927943  1 

0  1 

0  1 

0.53416734  1 

0  1 

-0.2599681  1 

0.57812141  1 

0  1 

0  1 

0  1 

0  1 

-0.2326459  1 

0  1 


SS 

0 

16.37701 

0.007068 

0.461255 

0.100753 

0.08897 

11.61315 

6.765115 

6.749429 

0.176749 

0.761108 

0.031519 

0.313561 

0.245723 

6.860332 

0.761108 

2.917298 

0.283114 

0.503607 

0.285685 

0.712849 

0.695275 

0.10211 

0.17891 

6.878601 

0.684827 

0.688505 

10.29449 

0.031657 

I. 669572 

II. 54136 
0.969921 
0.888755 
0.024013 
0.030527 
2.443065 
0.150536 


"Sig  Prob" 
0.0161 
0.0057 
0.0235 
0.0892 
0.1499 
0.1181 
0.0321 
0.0428 
0.0874 
0.1226 
0.2228 


Seq  SS 
9.500054 
11.62637 
7.280303 
3.935502 
2.769161 
3.205208 
5.824319 
4.963109 
3.408153 
2.71415 
1.669572 


RSquare 

0.0720 

0.1602 

0.2154 

0.2452 

0.2662 

0.2905 

0.3347 

0.3723 

0.3982 

0.4187 

0.4314 


"F  Ratio"  "Prob>F" 

0.000  1.0000 

14.850  0.0003 

0.006  0.9369 

0.415  0.5218 

0.090  0.7649 

0.080  0.7787 

10.530  0.0018 

6.134  0.0158 

6.120  0.0159 

0.158  0.6920 

0.687  0.4101 

0.028  0.8672 

0.281  0.5976 

0.220  0.6404 

6.221  0.0151 

0.687  0.4101 

2.645  0.1085 

0.254  0.6160 

0.453  0.5032 

0.256  0.6144 

0.643  0.4255 

0.627  0.4313 

0.091  0.7634 

0.160  0.6902 

6.237  0.0149 

0.617  0.4348 

0.621  0.4335 

9.335  0.0032 

0.028  0.8669 

1.514  0.2228 

10.465  0.0019 

0.878  0.3521 

0.804  0.3732 

0.021  0.8840 

0.027  0.8693 

2.215  0.1413 

0.135  0.7147 


Cp  p 

7.1782  2 

1 .2766  3 

-1.671  4 

-2.346  5 

-2.228  6 

-2.406  7 

-4.365  8 

-5.738  9 

-6.054  10 

-5.899  11 

-5.033  12 


Appendix  C 


Response  Column  59 
Actual  by  Predicted  Plot 


Column  59  Predicted  P<.0001  RSq=0.43 
RMSE=1.0502 


Summary  of  Fit 


RSquare  0.431397 

RSquare  Adj  0.339417 

Root  Mean  Square  Error  1.050151 

Mean  of  Response  4.4625 

Observations  (or  Sum  Wgts)  80 


Analysis  of  Variance 


Source  DF 

Model  1 1 

Error  68 

C.  Total  79 


Sum  of  Squares 
56.89590 
74.99160 
131.88750 


Mean  Square  F  Ratio 

5.17235  4.6901 

1.10282  Prob  >  F 

<.0001 


Lack  Of  Fit 


Source  DF 

Lack  Of  Fit  19 

Pure  Error  49 

Total  Error  68 


Sum  of  Squares 
28.389513 
46.602083 
74.991596 


Mean  Square  F  Ratio 

1.49418  1.5711 

0.95106  Prob  >  F 

0.1030 
Max  RSq 
0.6467 


Parameter  Estimates 

Term 

Intercept 

whattypesystemsavionics[1  -0] 
whattypeshrinkutilities[1  -0] 
whattypeshrinkinternet[1  -0] 
whattypecomponentdomain[1  -0] 
whattypeenterprisemanufact[1  -0] 
whattypeenterpriseOES[1  -0] 
whatprocessappsupptjl  -0] 
whatprocessdocumentationjl  -0] 
whatprocessfieldingjl  -0] 
whatprocessCMjl  -0] 
whatproductscommoncust[1  -0] 


Estimate 
4.3869501 
2.3092134 
2.6163426 
1.2592825 
-1.471877 
2.8668109 
-1.26007 
0.9855886 
-1 .068335 
0.5199362 
-1.156243 
0.4652918 


Std  Error 
0.163375 
0.599237 
0.806253 
0.508437 
0.594963 
1.149419 
0.77474 
0.394637 
0.349669 
0.422571 
0.357415 
0.312615 


t  Ratio  Prob>|t| 

26.85  <.0001 

3.85  0.0003 

3.25  0.0018 

2.48  0.0158 

-2.47  0.0159 

2.49  0.0151 

-1.63  0.1085 

2.50  0.0149 

-3.06  0.0032 

1 .23  0.2228 

-3.24  0.0019 

1.49  0.1413 


Effect  Tests 

Source 

whattypesystemsavionics 

whattypeshrinkutilities 

whattypeshrinkinternet 

whattypecomponentdomain 

whattypeenterprisemanufact 

whattypeenterpriseOES 

whatprocessappsuppt 

whatprocessdocumentation 

whatprocessfielding 

whatprocessCM 

whatproductscommoncust 


Nparm  DF 

1  1 

1  1 

1  1 

1  1 

1  1 

1  1 

1  1 

1  1 

1  1 

1  1 

1  1 


Sum  of  Squares 
16.377014 
11.613151 
6.765115 
6.749429 
6.860332 
2.917298 
6.878601 
10.294488 
1 .669572 
11.541363 
2.443065 


F  Ratio  Prob  >  F 

14.8502  0.0003 

10.5304  0.0018 

6.1344  0.0158 

6.1202  0.0159 

6.2207  0.0151 

2.6453  0.1085 

6.2373  0.0149 

9.3347  0.0032 

1.5139  0.2228 

10.4653  0.0019 

2.2153  0.1413 


AdminOverhead  =  4.39  +  sys-avia(2.31)  +  shrink-util(2.62)  +  shrink-int(1 .26)  +  comp-domain(-1.47)  +  ent-mnft(2.87)  +  ent- 
OES(-1.26)  +  proc-appsup(.99)  +  proc-doc(-1 .07)  +  proc-field(0.52)  +  proc-CM(-1.16)  +  prod-comcust(0.47) 
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Stepwise  Fit  -  Old  Survey  Data  -  Consequences  (ControlProcess) 

Response: 

Column  59 


Stepwise  Regression  Control 


Prob  to  Enter  0.250 

Prob  to  Leave  0.250 

Direction: 

Rules: 


Current  Estimates 

SSE  DFE 

115.04467  70 


MSE  RSquare 

1.6434952  0.3678 


RSquare  Adj 
0.2866 


Cp 

-8.705161 


AIC 

49.0635 


Lock 

X 


X  Intercept 

whattypesystemsavionics{0-1} 
whattypesystemsembedded{0-1 } 
whattypesystemscommunications{0-1} 
whattypesystemsdevice{0-1} 
whattypeshrinkbusiness{0-1} 

X  whattypeshrinkutilities{0-1} 

whattypeshrinkinternet{0-1 } 

X  whattypecomponentdomain{0-1} 

whattypecomponentCASE{0-1} 
whattypecomponentclass{0- 1 } 
whattypecomponentOS{0-1} 

X  whattypecomponentdevelopment{0-1} 

whattypeenterpriseacctng{0-1 } 
whattypeenterprisemanufact{0-1 } 
whattypeenterprisepayroll{0-1 } 
whattypeenterpriseOES{0- 1 } 
whattypeenterprisescripting{0-1} 
whattypeenterpriseweb{0-1 } 
whatprocessrequirements{0-1} 

X  whatprocessdesign{0-1} 

whatprocesstesting{0-1} 
whatprocessmaintenance{0-1} 

X  whatprocessreengineering{0-1} 

whatprocessappsuppt{0-1 } 
whatprocesstraining{0-1} 

X  whatprocessspecification{0-1} 

whatprocessdocumentation{0-1 } 
whatprocesscoding{0- 1 } 
whatprocessfielding{0-1} 
whatprocessCM{0-1} 

X  whatprocesstoolsuppt{0-1} 

whatprocessS  WEngSuppt{0- 1 } 
whatproductscustom{0-1 } 

X  whatproductsCOTS{0-1} 

X  whatproductscommoncust{0-1} 

whatproductsnone{0-1 } 


Step  History 

Step  Parameter 

1  whatprocessdesign{0-1} 

2  whatprocessreengineering{0-1} 

3  whatproductscommoncust{0-1} 

4  whattypecomponentdevelopment{0- 1 } 

5  whatprocesstoolsuppt{0-1} 

6  whattypecomponentdomain{0-1} 

7  whatprocessspecification{0-1} 

8  whatproductsCOTS{0-1} 

9  whattypeshrinkutilities{0-1} 


Action 

Entered 

Entered 

Entered 

Entered 

Entered 

Entered 

Entered 

Entered 

Entered 


Estimate  nDF 

4.03142772  1 

0  1 

0  1 

0  1 

0  1 

0  1 

-0.9435264  1 

0  1 

0.6251688  1 

0  1 

0  1 

0  1 

0.34640918  1 

0  1 

0  1 

0  1 

0  1 

0  1 

0  1 

0  1 

0.6918918  1 

0  1 

0  1 

-0.7477925  1 

0  1 

0  1 

-0.2610422  1 

0  1 

0  1 

0  1 

0  1 

0.33341554  1 

0  1 

0  1 

0.40698632  1 

0.23074774  1 

0  1 


SS 

0 

2.074114 
0.166172 
0.006024 
0.107994 
0.01269 
5.266053 
0.31135 
6.437607 
0.260637 
0.216523 
0.525894 
5.126021 
0.094156 
0.047465 
0.216523 
0.318245 
0.960608 
0.071601 
0.830574 
20.57616 
1.967108 
1 .28478 
20.08407 
1.847033 
0.600308 
3.014046 
0.289843 
1 .656468 
1.311587 
1.107612 
4.539314 
0.416103 
0.181642 
5.855804 
2.398779 
0.489627 


"F  Ratio" 
0.000 
1.267 
0.100 
0.004 
0.065 
0.008 
3.204 
0.187 
3.917 
0.157 
0.130 
0.317 
3.119 
0.057 
0.028 
0.130 
0.191 
0.581 
0.043 
0.502 
12.520 
1.200 
0.779 
12.220 
1.126 
0.362 
1.834 
0.174 
1.008 
0.796 
0.671 
2.762 
0.250 
0.109 
3.563 
1.460 
0.295 


'Prob>F" 

1.0000 

0.2643 

0.7530 

0.9522 

0.7998 

0.9307 

0.0778 

0.6666 

0.0517 

0.6935 

0.7194 

0.5753 

0.0817 

0.8128 

0.8665 

0.7194 

0.6631 

0.4485 

0.8364 

0.4811 

0.0007 

0.2771 

0.3804 

0.0008 

0.2924 

0.5494 

0.1800 

0.6776 

0.3189 

0.3755 

0.4156 

0.1010 

0.6183 

0.7422 

0.0632 

0.2311 

0.5888 


'Sig  Prob"  Seq  SS 

0.0032  19.33694 

0.0025  18.34199 

0.0395  7.880786 

0.1523  3.701568 

0.1863  3.116936 

0.1904  3.028647 

0.1685  3.313473 

0.1908  2.956437 

0.0778  5.266053 


RSquare  Cp  p 

0.1063  -3.479  2 

0.2070  -9.657  3 

0.2503  -11.17  4 

0.2707  -10.82  5 

0.2878  -10.21  6 

0.3045  -9.562  7 

0.3227  -9.039  8 

0.3389  -8.357  9 

0.3678  -8.705  10 


Appendix  C 


Response  Column  59 
Actual  by  Predicted  Plot 

8n - 


- 1 - 1 - 1 - 1 - 1 - 

1  2  3  4  5  6  7 

Column  59  Predicted  P=0.0001  RSq=0.37 
RMSE= 1.282 


Summary  of  Fit 


RSquare  0.367843 

RSquare  Adj  0.286566 

Root  Mean  Square  Error  1.281989 

Mean  of  Response  3.4875 

Observations  (or  Sum  Wgts)  80 


Analysis  of  Variance 


Source  DF 

Sum  of  Squares  Mean  Square 

F  Ratio 

Model  9 

66.94283 

7.43809 

4.5258 

Error  70 

115.04467 

1.64350 

Prob  >  F 

C.  Total  79 

181.98750 

0.0001 

Lack  Of  Fit 

Source  DF 

Sum  of  Squares 

Mean  Square 

F  Ratio 

Lack  Of  Fit  24 

53.62800 

2.23450 

1.6736 

Pure  Error  46 

61.41667 

1.33514 

Prob  >  F 

Total  Error  70 

115.04467 

0.0662 

Max  RSq 

0.6625 

Parameter  Estimates 

Term 

Estimate 

Std  Error 

t  Ratio  Prob>|t| 

Intercept 

4.7136861 

0.362359 

13.01  <.0001 

whattypeshrinkutilities[1-0] 

1.8870527 

1 .054206 

1.79  0.0778 

whattypecomponentdomain[1  -0] 

-1.250338 

0.631756 

-1.98  0.0517 

whattypecomponentdevelopment[1-0] 

-0.692818 

0.392296 

-1.77  0.0817 

whatprocessdesign[1  -0] 

-1 .383784 

0.391084 

-3.54  0.0007 

whatprocessreengineering[1-0] 

1.4955849 

0.427828 

3.50  0.0008 

whatprocessspecificationjl  -0] 

0.5220845 

0.385523 

1.35  0.1800 

whatprocesstoolsuppt[1  -0] 

-0.666831 

0.401241 

-1.66  0.1010 

whatproductsCOTSjl  -0] 

-0.813973 

0.431222 

-1.89  0.0632 

whatproductscommoncust[1  -0] 

-0.461495 

0.381994 

-1.21  0.2311 

Effect  Tests 

Source 

Nparm 

DF 

Sum  of  Squares 

F  Ratio 

Prob  >  F 

whattypeshrinkutilities 

1 

1 

5.266053 

3.2042 

0.0778 

whattypecomponentdomain 

1 

1 

6.437607 

3.9170 

0.0517 

whattypecomponentdevelopment 

1 

1 

5.126021 

3.1190 

0.0817 

whatprocessdesign 

1 

1 

20.576158 

12.5198 

0.0007 

whatprocessreengineering 

1 

1 

20.084070 

12.2203 

0.0008 

whatprocessspecification 

1 

1 

3.014046 

1.8339 

0.1800 

whatprocesstoolsuppt 

1 

1 

4.539314 

2.7620 

0.1010 

whatproductsCOTS 

1 

1 

5.855804 

3.5630 

0.0632 

whatproductscommoncust 

1 

1 

2.398779 

1.4596 

0.2311 

ControlProcess  =  4.71  +  shrink-util(1 .89)  +  comp-domain(-1 .25)  +  comp-dev(-0.69)  +proc-des(-1 .38)  +  proc-reeng(1.50)  + 
proc-spec(0.52)  +  proc-toolsup(-0.67)  +  prod-COTS(-0.81)  +  prod-comcust(-0.46) 
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Stepwise  Fit  -  Old  Survey  Data  -  Consequences  (InhouseNonCore) 

Response: 

Column  59 


Stepwise  Regression  Control 

Prob  to  Enter  0.250 

Prob  to  Leave  0.250 

Direction: 

Rules: 


Current  Estimates 

SSE  DFE 

76.23804  67 


MSE  RSquare 

1.1378812  0.3996 


RSquare  Adj 
0.3100 


Cp  AIC 

-4.042264  20.21783 


Entered  Parameter 
X  Intercept 

X  whattypesystemsavionics{0-1} 

whattypesystemsembedded{0-1 } 
whattypesystemscommunications{0-1} 
whattypesystemsdevice{0- 1 } 
whattypeshrinkbusiness{0-1} 
whattypeshrinkutilities{0-1 } 
whattypeshrinkinternet{0-1 } 
whattypecomponentdomain{0-1} 
whattypecomponentCASE{0-1 } 
whattypecomponentclass{0- 1 } 
whattypecomponentOS{0-1 } 

X  whattypecomponentdevelopment{0-1} 

X  whattypeenterpriseacctng{0-1} 

X  whattypeenterprisemanufact{0-1} 

whattypeenterprisepayroll{0-1} 

X  whattypeenterpriseOES{0-1} 

whattypeenterprisescripting{0-1} 

X  whattypeenterpriseweb{0-1} 

whatprocessrequirements{0-1} 
whatprocessdesign{0- 1 } 
whatprocesstesting{0- 1 } 
whatprocessmaintenance{0-1} 
whatprocessreengineering{0-1} 
whatprocessappsuppt{0-1 } 
whatprocesstraining{0-1} 
whatprocessspecification{0-1 } 
whatprocessdocumentation{0-1 } 
whatprocesscoding{0- 1 } 
whatprocessfielding{0-1} 
whatprocessCM{0-1} 

X  whatprocesstoolsuppt{0-1} 

X  whatprocessSWEngSuppt{0-1} 

whatproductscustom{0-1 } 

X  whatproductsCOTS{0-1} 

whatproductscommoncust{0-1} 
whatproductsnone{0-1 } 


Step  History 

Step  Parameter 


10 

11 

12 


whattypeenterpriseweb{0-1 } 

whatproductsCOTS{0-1} 

whattypeenterprisemanufact{0-1} 

whatprocessfielding{0-1} 

whattypesystemsavionics{0-1} 

whatprocessSWEngSuppt{0-1} 

whatprocesstoolsuppt{0-1} 

whatproductsnone{0-1} 

whattypeenterpriseOES{0-1} 

whattypeenterpriseacctng{0-1} 

whatprocessfielding{0-1 } 

whattypecomponentdevelopment{0-1} 


Estimate  nDF 

4.05266848  1 

0.60304818  1 

0  1 

0  1 

0  1 

0  1 

0  1 

0  1 

0  1 

0  1 

0  1 

0  1 

-0.2397377  1 

-0.4283596  1 

-1.1718055  1 

0  1 

0.90718493  1 

0  1 

-0.3081043  1 

0  1 

0  1 

0  1 

0  1 

0  1 

0  1 

0  1 

0  1 

0  1 

0  1 

0  1 

0  1 

0.41502599  1 

-0.4251425  1 

0  1 

-0.3367  1 

0  1 

0.56807799  1 


SS 

0 

5.955562 

1.112126 

0.094749 

0.129358 

0.77388 

1.395702 

0.751067 

0.69316 

0.075683 

0.000966 

0.050639 

2.287103 

2.999612 

5.048142 

0.000966 

5.577901 

0.411367 

3.008214 

0.002677 

0.292839 

0.579725 

0.009492 

0.00165 

0.594047 

0.777955 

0.251669 

0.048766 

0.127047 

1.263076 

0.161346 

6.432741 

8.94256 

0.460933 

4.988808 

1.234165 

2.427014 


"F  Ratio" 
0.000 
5.234 
0.977 
0.082 
0.112 
0.677 
1.231 
0.657 
0.606 
0.066 
0.001 
0.044 
2.010 
2.636 
4.436 
0.001 
4.902 
0.358 
2.644 
0.002 
0.254 
0.506 
0.008 
0.001 
0.518 
0.680 
0.219 
0.042 
0.110 
1.112 
0.140 
5.653 
7.859 
0.401 
4.384 
1.086 
2.133 


'Prob>F" 

1.0000 

0.0253 

0.3265 

0.7753 

0.7387 

0.4136 

0.2713 

0.4206 

0.4392 

0.7987 

0.9770 

0.8347 

0.1609 

0.1092 

0.0389 

0.9770 

0.0302 

0.5516 

0.1087 

0.9617 

0.6156 

0.4795 

0.9280 

0.9700 

0.4741 

0.4124 

0.6417 

0.8378 

0.7410 

0.2955 

0.7095 

0.0203 

0.0066 

0.5285 

0.0401 

0.3012 

0.1488 


Action 

Entered 

Entered 

Entered 

Entered 

Entered 

Entered 

Entered 

Entered 

Entered 

Entered 

Removed 

Entered 


'Sig  Prob"  Seq  SS 

0.0121  10.15385 

0.0256  7.554217 

0.0501  5.558983 

0.0716  4.540449 

0.0743  4.322517 

0.0448  5.264054 

0.0544  4.645265 

0.1157  3.014268 

0.1774  2.178614 

0.1442  2.51728 

0.2944  1.287458 

0.1609  2.287103 


RSquare  Cp  p 

0.0800  5.6242  2 

0.1394  2.4759  3 

0.1832  0.6873  4 

0.2190  -0.407  5 

0.2530  -1.353  6 

0.2945  -2.941  7 

0.3311  -4.106  8 

0.3548  -4.161  9 

0.3719  -3.645  10 

0.3918  -3.361  11 

0.3816  -4.484  10 

0.3996  -4.042  1 1 


Appendix  C 


Response  Column  59 
Actual  by  Predicted  Plot 


Column  59  Predicted  P<.0001  RSq=0.40 
RMSE=1 .0667 


Summary  of  Fit 


RSquare  0.39964 

RSquare  Adj  0.310034 

Root  Mean  Square  Error  1.066715 

Mean  of  Response  3.987179 

Observations  (or  Sum  Wgts)  78 


Analysis  of  Variance 

Source 

DF 

Sum  of  Squares 

Mean  Square 

Model 

10 

50.74914 

5.07491 

Error 

67 

76.23804 

1.13788 

C.  Total 

77 

126.98718 

F  Ratio 
4.4600 
Prob  >  F 
<.0001 


Lack  Of  Fit 


Source  DF 

Lack  Of  Fit  16 

Pure  Error  51 

Total  Error  67 


Sum  of  Squares 
20.952326 
55.285714 
76.238040 


Mean  Square 
1.30952 
1.08403 


F  Ratio 
1.2080 
Prob  >  F 
0.2944 
Max  RSq 
0.5646 


Parameter  Estimates 

Term 

Intercept 

whattypesystemsavionics[1  -0] 
whattypecomponentdevelopment[1-0] 
whattypeenterpriseacctng[1  -0] 
whattypeenterprisemanufact[1  -0] 
whattypeenterpriseOES[1  -0] 
whattypeenterpriseweb[1-0] 
whatprocesstoolsuppt[1  -0] 
whatprocessSWEngSuppt[1  -0] 
whatproductsCOTSjl  -0] 
whatproductsnone[1  -0] 


Estimate 
3.636156 
-1 .206096 
0.4794754 
0.8567193 
2.343611 
-1.81437 
0.6162086 
-0.830052 
0.850285 
0.6733999 
-1.136156 


Std  Error 
0.190425 
0.527192 
0.338199 
0.52766 
1.112675 
0.819481 
0.378985 
0.349105 
0.303307 
0.321605 
0.777947 


t  Ratio  Prob>|t| 

19.09  <.0001 

-2.29  0.0253 

1.42  0.1609 

1.62  0.1092 

2.11  0.0389 

-2.21  0.0302 

1.63  0.1087 

-2.38  0.0203 

2.80  0.0066 

2.09  0.0401 

-1.46  0.1488 


Effect  Tests 

Source 

whattypesystemsavionics 

whattypecomponentdevelopment 

whattypeenterpriseacctng 

whattypeenterprisemanufact 

whattypeenterpriseOES 

whattypeenterpriseweb 

whatprocesstoolsuppt 

whatprocessSWEngSuppt 

whatproductsCOTS 

whatproductsnone 


Nparm  DF 

1  1 

1  1 

1  1 

1  1 

1  1 

1  1 

1  1 

1  1 

1  1 

1  1 


Sum  of  Squares 
5.9555625 
2.2871033 
2.9996120 
5.0481421 
5.5779009 
3.0082137 
6.4327412 
8.9425598 
4.9888077 
2.4270141 


F  Ratio  Prob  >  F 

5.2339  0.0253 

2.0100  0.1609 

2.6361  0.1092 

4.4364  0.0389 

4.9020  0.0302 

2.6437  0.1087 

5.6533  0.0203 

7.8590  0.0066 

4.3843  0.0401 

2.1329  0.1488 


InhouseNonCore  =  3.64  +  sys-avia(-1.21)  +  comp-dev(0.48)  +  ent-acct(0.86)  +  ent-mnft(2.34)  +  ent-OES(-1.81)  +  ent- 
web(0.62)  +  proc-toolsup(-0.83)  +  proc-SWEngSup(0.85)  +  prod-COTS(0.67)  +  prod-none(-1.14) 
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Stepwise  Fit  -  Old  Survey  Data  -  Consequences  (InhouseTurnover) 

Response: 

Column  59 


Stepwise  Regression  Control 

Prob  to  Enter  0.250 

Prob  to  Leave  0.250 

Direction: 

Rules: 


Current  Estimates 

SSE  DFE 

20.108654  67 


MSE  RSquare 

0.3001292  0.4236 


RSquare  Adj 
0.3290 


Cp  AIC 

-1.917523  -84.0955 


Lock 

X 


X  Intercept 

X  whattypesystemsavionics{0-1} 

whattypesystemsembedded{0-1 } 
whattypesystemscommunications{0-1} 
X  whattypesystemsdevice{0-1} 

X  whattypeshrinkbusiness{0-1} 

whattypeshrinkutilities{0-1} 
whattypeshrinkinternet{0-1 } 
whattypecomponentdomain{0-1} 

X  whattypecomponentCASE{0-1} 

whattypecomponentclass{0- 1 } 
whattypecomponentOS{0-1} 
whattypecomponentdevelopment{0-1} 
X  whattypeenterpriseacctng{0-1} 

whattypeenterprisemanufact{0-1 } 
whattypeenterprisepayroll{0-1 } 

X  whattypeenterpriseOES{0-1} 

X  whattypeenterprisescripting{0-1} 

whattypeenterpriseweb{0-1 } 
whatprocessrequirements{0-1} 

X  whatprocessdesign{0-1} 

whatprocesstesting{0-1} 

X  whatprocessmaintenance{0-1} 

whatprocessreengineering{0-1} 
whatprocessappsuppt{0-1 } 
whatprocesstraining{0-1} 
whatprocessspecification{0-1 } 
whatprocessdocumentation{0-1 } 
whatprocesscoding{0- 1 } 
whatprocessfielding{0-1} 

X  whatprocessCM{0-1} 

whatprocesstoolsuppt{0-1} 

X  whatprocessSWEngSuppt{0-1} 

whatproductscustom{0-1 } 
whatproductsCOTS{0-1} 
whatproductscommoncust{0-1} 
whatproductsnone{0-1 } 


Step  History 

Step  Parameter 

1  whattypecomponentCASE{0-1} 

2  whatprocessSWEngSuppt{0-1} 

3  whattypesystemsdevice{0-1} 

4  whattypeshrinkbusiness{0-1} 

5  whattypeenterpriseacctng{0-1} 

6  whatprocessmaintenance{0-1} 

7  whattypeenterprisescripting{0-1} 

8  whattypesystemsavionics{0-1} 

9  whatprocessCM{0-1} 

10  whattypeenterpriseOES{0-1} 

11  whatprocessdesign{0-1} 


Action 

Entered 

Entered 

Entered 

Entered 

Entered 

Entered 

Entered 

Entered 

Entered 

Entered 

Entered 


Estimate  nDF 

2.95393134  1 

0.22717178  1 

0  1 

0  1 

0.21799118  1 

0.26796463  1 

0  1 

0  1 

0  1 

0.44251331  1 

0  1 

0  1 

0  1 

-0.3015485  1 

0  1 

0  1 

0.32671654  1 

0.21714431  1 

0  1 

0  1 

0.10946224  1 

0  1 

0.17301282  1 

0  1 

0  1 

0  1 

0  1 

0  1 

0  1 

0  1 

-0.1637265  1 

0  1 

-0.1579166  1 

0  1 

0  1 

0  1 

0  1 


SS 

0 

0.84964 

0.105511 

0.096503 

0.68082 

2.111955 

0.076623 

0.001559 

0.045673 

1.705531 

0.001236 

0.003064 

0.223144 

1.518858 

0.165526 

0.001236 

0.733246 

0.616033 

0.02785 

0.107489 

0.548418 

0.161196 

1.621516 

0.030023 

0.371469 

0.147712 

0.036901 

0.058968 

0.127897 

0.16686 

0.970396 

0.053137 

1.360082 

0.017059 

0.011916 

0.054486 

0.308644 


"Sig  Prob" 
0.0002 
0.0491 
0.0530 
0.0674 
0.0485 
0.1051 
0.1746 
0.2280 
0.1689 
0.1704 
0.1810 


Seq  SS 
5.864146 
1.450896 
1.351052 
1.166644 
1.30977 
0.856237 
0.590655 
0.461396 
0.595006 
0.583203 
0.548418 


RSquare 

0.1681 

0.2097 

0.2484 

0.2819 

0.3194 

0.3439 

0.3609 

0.3741 

0.3912 

0.4079 

0.4236 


"F  Ratio"  "Prob>F" 

0.000  1.0000 

2.831  0.0971 

0.348  0.5572 

0.318  0.5746 

2.268  0.1367 

7.037  0.0100 

0.252  0.6170 

0.005  0.9432 

0.150  0.6995 

5.683  0.0200 

0.004  0.9494 

0.010  0.9204 

0.741  0.3926 

5.061  0.0278 

0.548  0.4618 

0.004  0.9494 

2.443  0.1228 

2.053  0.1566 

0.092  0.7632 

0.355  0.5535 

1.827  0.1810 

0.533  0.4678 

5.403  0.0231 

0.099  0.7544 

1.242  0.2691 

0.488  0.4871 

0.121  0.7287 

0.194  0.6610 

0.422  0.5180 

0.552  0.4600 

3.233  0.0767 

0.175  0.6772 

4.532  0.0370 

0.056  0.8136 

0.039  0.8438 

0.179  0.6733 

1.029  0.3141 


Cp  p 

1.6116  2 

-0.218  3 

-1.785  4 

-2.865  5 

-4.322  6 

-4.582  7 

-4.142  8 

-3.36  9 

-2.93  10 

-2.47  1 1 

-1.918  12 


Appendix  C 


Response  Column  59 
Actual  by  Predicted  Plot 

I  8 

7 

I6 

<  5 

c4 

I3 

O  2 
1 
0 

1  2  3  4  5  6  7 

Column  59  Predicted  P<.0001  RSq=0.42 
RMSE=0.5478 


Summary  of  Fit 

0.423591 
0.328956 
0.54784 
4.037975 
79 


RSquare 
RSquare  Adj 
Root  Mean  Square  Error 
Mean  of  Response 
Observations  (or  Sum  Wgts) 


Analysis  of  Variance 


Source  DF 

Model  1 1 

Error  67 

C.  Total  78 


Sum  of  Squares 
14.777422 
20.108654 
34.886076 


Mean  Square  F  Ratio 

1 .34340  4.4761 

0.30013  Prob  >  F 

<.0001 


Lack  Of  Fit 


Source  DF 

Lack  Of  Fit  21 

Pure  Error  46 

Total  Error  67 


Sum  of  Squares 
9.999130 
10.109524 
20.108654 


Mean  Square  F  Ratio 

0.476149  2.1666 

0.219772  Prob  >  F 

0.0145 
Max  RSq 
0.7102 


Parameter  Estimates 

Term 

Intercept 

whattypesystemsavionics[1  -0] 
whattypesystemsdevice[1  -0] 
whattypeshrinkbusiness[1  -0] 
whattypecomponentCASE[1  -0] 
whattypeenterpriseacctng[1  -0] 
whattypeenterpriseOES[1  -0] 
whattypeenterprisescripting[1  -0] 
whatprocessdesign[1-0] 
whatprocessmaintenance[1  -0] 
whatprocessCM[1  -0] 
whatprocessSWEngSuppt[1  -0] 


Estimate 

4.3127166 

-0.454344 

-0.435982 

-0.535929 

-0.885027 

0.603097 

-0.653433 

-0.434289 

-0.218924 

-0.346026 

0.3274531 

0.3158332 


Std  Error 
0.146456 
0.270036 
0.289472 
0.202032 
0.371262 
0.268091 
0.418052 
0.303131 
0.161954 
0.148868 
0.182108 
0.148364 


t  Ratio 
29.45 
-1.68 
-1.51 
-2.65 
-2.38 
2.25 
-1.56 
-1.43 
-1.35 
-2.32 
1.80 
2.13 


Prob>|t| 

<.0001 

0.0971 

0.1367 

0.0100 

0.0200 

0.0278 

0.1228 

0.1566 

0.1810 

0.0231 

0.0767 

0.0370 


Effect  Tests 

Source 

whattypesystemsavionics 

whattypesystemsdevice 

whattypeshrinkbusiness 

whattypecomponentCASE 

whattypeenterpriseacctng 

whattypeenterpriseOES 

whattypeenterprisescripting 

whatprocessdesign 

whatprocessmaintenance 

whatprocessCM 

whatprocessSWEngSuppt 


Nparm  DF 

1  1 

1  1 

1  1 

1  1 

1  1 

1  1 

1  1 

1  1 

1  1 

1  1 

1  1 


Sum  of  Squares 
0.8496399 
0.6808203 
2.1119545 
1.7055312 
1.5188578 
0.7332455 
0.6160329 
0.5484179 
1.6215155 
0.9703965 
1.3600824 


F  Ratio  Prob  >  F 

2.8309  0.0971 

2.2684  0.1367 

7.0368  0.0100 

5.6827  0.0200 

5.0607  0.0278 

2.4431  0.1228 

2.0526  0.1566 

1.8273  0.1810 

5.4027  0.0231 

3.2333  0.0767 

4.5317  0.0370 


InhouseTurnover  =  4.31  +  sys-avia(-0.45)  +  sys-dev(-0.44)  +  shrink-bus(-0.54)  +  comp-CASE(-0.89)  +  ent-acct(0.60)  + 
ent-OES(-0.65)  +  ent-script(-0.43)  +  proc-des(-0.22)  +  proc-maint(-0.35)  +  proc-CM(0.33)  +  proc-SWEngSup(0.32) 
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Stepwise  Fit  -  Old  Survey  Data  -  Consequences  (LearningCurve) 

Response: 

Column  59 


Stepwise  Regression  Control 

Prob  to  Enter  0.250 

Prob  to  Leave  0.250 

Direction: 

Rules: 


Current  Estimates 

SSE  DFE 

80.893033  72 


MSE  RSquare 

1.1235143  0.2032 


RSquare  Adj 
0.1368 


Cp  AIC 

-12.97511  15.87071 


Lock 

X 


X  Intercept 

whattypesystemsavionics{0-1} 
whattypesystemsembedded{0-1 } 
whattypesystemscommunications{0-1} 
whattypesystemsdevice{0-1} 
whattypeshrinkbusiness{0-1} 
whattypeshrinkutilities{0-1 } 
whattypeshrinkinternet{0-1 } 
whattypecomponentdomain{0-1} 

X  whattypecomponentCASE{0-1} 

X  whattypecomponentclass{0-1} 

whattypecomponentOS{0-1} 
whattypecomponentdevelopment{0-1} 
whattypeenterpriseacctng{0-1 } 
whattypeenterprisemanufact{0-1 } 
whattypeenterprisepayroll{0-1 } 

X  whattypeenterpriseOES{0-1} 

X  whattypeenterprisescripting{0-1} 

whattypeenterpriseweb{0-1 } 
whatprocessrequirements{0-1} 
whatprocessdesign{0- 1 } 
whatprocesstesting{0-1} 

X  whatprocessmaintenance{0-1} 

whatprocessreengineering{0-1} 
whatprocessappsuppt{0-1 } 
whatprocesstraining{0-1} 
whatprocessspecification{0-1 } 
whatprocessdocumentation{0-1 } 
whatprocesscod  i  ng{0- 1 } 
whatprocessfielding{0-1} 
whatprocessCM{0-1} 
whatprocesstoolsuppt{0-1} 
whatprocessSWEngSuppt{0- 1 } 
whatproductscustom{0-1 } 
whatproductsCOTS{0-1} 

X  whatproductscommoncust{0-1} 
whatproductsnone{0-1 } 


Step  History 

Step  Parameter 

1  whattypecomponentclass{0-1} 

2  whattypecomponentCASE{0-1} 

3  whatprocessreengineering{0-1} 

4  whattypeenterprisescripting{0-1} 

5  whatproductscommoncust{0-1} 

6  whatprocessmaintenance{0-1} 

7  whatprocessreengineering{0-1} 

8  whattypeenterpriseOES{0-1} 


Action 

Entered 

Entered 

Entered 

Entered 

Entered 

Entered 

Removed 

Entered 


Estimate  nDF 

2.71713895  1 

0  1 

0  1 

0  1 

0  1 

0  1 

0  1 

0  1 

0  1 

0.77915246  1 

0.7420407  1 

0  1 

0  1 

0  1 

0  1 

0  0 

0.56033792  1 

-0.4359535  1 

0  1 

0  1 

0  1 

0  1 

0.18170279  1 

0  1 

0  1 

0  1 

0  1 

0  1 

0  1 

0  1 

0  1 

0  1 

0  1 

0  1 

0  1 

-0.2767501  1 

0  1 


"Sig  Prob" 
0.0343 
0.0525 
0.1175 
0.1551 
0.1640 
0.2405 
0.3951 
0.2046 


Seq  SS 
5.772234 
4.652159 
2.946976 
2.391469 
2.260892 
1.593321 
0.832478 
1.841381 


SS 

0 

0.00014 
0.061453 
0.010547 
0.046311 
0.38614 
0.273315 
0.048634 
0.259912 
6.426484 
3.142977 
0.725277 
0.239404 
1 .445996 
0.223971 
0 

1.841381 

2.614876 

0.004971 

0.06517 

0.790721 

0.003366 

2.200086 

0.455925 

0.035675 

0.008131 

0.588197 

0.178102 

0.747153 

0.049553 

0.048057 

0.00153 

0.024241 

1.179177 

0.369487 

4.097259 

1.245628 


RSquare 

0.0569 

0.1027 

0.1317 

0.1553 

0.1775 

0.1932 

0.1850 

0.2032 


"F  Ratio" 
0.000 
0.000 
0.054 
0.009 
0.041 
0.341 
0.241 
0.043 
0.229 
5.720 
2.797 
0.642 
0.211 
1.292 
0.197 

1.639 

2.327 

0.004 

0.057 

0.701 

0.003 

1.958 

0.402 

0.031 

0.007 

0.520 

0.157 

0.662 

0.044 

0.042 

0.001 

0.021 

1.050 

0.326 

3.647 

1.110 


Cp 

-13.42 

-14.41 

-14.31 

-13.85 

-13.3 

-12.33 

-13.79 

-12.98 


'Prob>F" 

1.0000 

0.9912 

0.8169 

0.9236 

0.8408 

0.5614 

0.6252 

0.8369 

0.6338 

0.0194 

0.0988 

0.4255 

0.6476 

0.2595 

0.6584 

0.2046 

0.1315 

0.9475 

0.8116 

0.4053 

0.9568 

0.1660 

0.5279 

0.8600 

0.9329 

0.4732 

0.6934 

0.4186 

0.8353 

0.8378 

0.9709 

0.8844 

0.3089 

0.5700 

0.0602 

0.2956 


P 

2 

3 

4 

5 

6 
7 
6 
7 


Appendix  C 


Response  Column  59 
Actual  by  Predicted  Plot 


i  i  i  i  i 

1  2  3  4  5  6  7 

Column  59  Predicted  P=0.0101  RSq=0.20 
RMSE=1 .06 


Summary  of  Fit 


RSquare  0.203173 

RSquare  Adj  0.136771 

Root  Mean  Square  Error  1 .05996 

Mean  of  Response  4.177215 

Observations  (or  Sum  Wgts)  79 


Analysis  of  Variance 


Source  DF 

Model  6 

Error  72 

C.  Total  78 


Sum  of  Squares 
20.62595 
80.89303 
101.51899 


Mean  Square  F  Ratio 

3.43766  3.0597 

1.12351  Prob  >  F 

0.0101 


Lack  Of  Fit 

Source 
Lack  Of  Fit 
Pure  Error 
Total  Error 


DF  Sum  of  Squares 

3  0.839604 

69  80.053429 

72  80.893033 


Mean  Square  F  Ratio 

0.27987  0.2412 

1.16019  Prob  >  F 

0.8673 
Max  RSq 
0.2114 


Parameter  Estimates 

Term 

Intercept 

whattypecomponentCASE[1  -0] 
whattypecomponentclass[1  -0] 
whattypeenterpriseOES[1  -0] 
whattypeenterprisescripting[1  -0] 
whatprocessmaintenance[1  -0] 
whatproductscommoncust[1  -0] 


Estimate 

4.2676693 

-1.558305 

-1.484081 

-1.120676 

0.871907 

-0.363406 

0.5535001 


Std  Error 
0.173822 
0.651561 
0.887312 
0.875382 
0.571523 
0.259693 
0.289841 


t  Ratio 
24.55 
-2.39 
-1.67 
-1.28 
1.53 
-1.40 
1.91 


Prob>|t| 

<.0001 

0.0194 

0.0988 

0.2046 

0.1315 

0.1660 

0.0602 


Effect  Tests 

Source  Nparm  DF 

whattypecomponentCASE  1  1 

whattypecomponentclass  1  1 

whattypeenterpriseOES  1  1 

whattypeenterprisescripting  1  1 

whatprocessmaintenance  1  1 

whatproductscommoncust  1  1 


Sum  of  Squares  F  Ratio  Prob  >  F 

6.4264844  5.7200  0.0194 

3.1429766  2.7975  0.0988 

1.8413814  1.6389  0.2046 

2.6148756  2.3274  0.1315 

2.2000860  1.9582  0.1660 

4.0972593  3.6468  0.0602 


LearningCurve  =  4.27  +  comp-CASE(-1 .56)  +  comp-class(-1 .48)  +  ent-OES(-1.12)  +  ent-script(0.87)  +  proc-maint(-0.36)  + 
prod-comcus(0.55) 
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Stepwise  Fit  -  Old  Survey  Data  -  Consequences  (Risk) 

Response: 

Column  59 


Stepwise  Regression  Control 

Prob  to  Enter  0.250 

Prob  to  Leave  0.250 

Direction: 

Rules: 


Current  Estimates 

SSE  DFE 

133.27569  72 


MSE  RSquare 

1.8510512  0.3065 


RSquare  Adj 
0.2391 


Cp  AIC 

-5.483647  56.83146 


Lock 

X 


X  Intercept 

whattypesystemsavionics{0-1} 
whattypesystemsembedded{0-1 } 

X  whattypesystemscommunications{0-1} 

whattypesystemsdevice{0-1} 
whattypeshrinkbusiness{0-1} 
whattypeshrinkutilities{0-1 } 
whattypeshrinkinternet{0-1 } 
whattypecomponentdomain{0-1} 
whattypecomponentCASE{0-1} 
whattypecomponentclass{0- 1 } 
whattypecomponentOS{0-1} 
whattypecomponentdevelopment{0-1} 
whattypeenterpriseacctng{0-1 } 
whattypeenterprisemanufact{0-1 } 
whattypeenterprisepayroll{0-1 } 

X  whattypeenterpriseOES{0-1} 

X  whattypeenterprisescripting{0-1} 

X  whattypeenterpriseweb{0-1} 

whatprocessrequirements{0-1} 
whatprocessdesign{0- 1 } 
whatprocesstesting{0- 1 } 
whatprocessmaintenance{0-1} 
whatprocessreengineering{0-1} 
whatprocessappsuppt{0-1 } 
whatprocesstraining{0-1} 
whatprocessspecification{0-1 } 
whatprocessdocumentation{0-1 } 
whatprocesscoding{0- 1 } 
whatprocessfielding{0-1} 

X  whatprocessCM{0-1} 

whatprocesstoolsuppt{0-1} 

X  whatprocessSWEngSuppt{0-1} 

whatproductscustom{0-1 } 

X  whatproductsCOTS{0-1} 

whatproductscommoncust{0-1} 
whatproductsnone{0-1 } 


Step  History 

Step  Parameter 

1  whatproductsCOTS{0-1} 

2  whatprocessCM{0-1} 

3  whattypesystemscommunications{0-1 } 

4  whatprocessSWEngSuppt{0-1} 

5  whattypeenterprisescripting{0-1 } 

6  whattypeenterpriseOES{0-1} 

7  whattypeenterpriseweb{0-1} 


Estimate  nDF 

2.42730174  1 

0  1 

0  1 

0.50306595  1 

0  1 

0  1 

0  1 

0  1 

0  1 

0  1 

0  1 

0  1 

0  1 

0  1 

0  1 

0  1 

0.81973707  1 

0.63294739  1 

0.26732438  1 

0  1 

0  1 

0  1 

0  1 

0  1 

0  1 

0  1 

0  1 

0  1 

0  1 

0  1 

0.69310296  1 

0  1 

-0.3151494  1 

0  1 

-0.5109024  1 

0  1 

0  1 


SS 

0 

0.420308 
0.714094 
1 1 .67243 
0.707423 
0.429136 
1.016441 
0.001256 
0.056183 
2.26576 
0.258155 
0.003651 
0.937601 
0.808722 
1 .667474 
0.258155 
5.073979 
5.777588 
2.901482 
0.117853 
0.567151 
0.389428 
0.001993 
0.215168 
0.755874 
1.214825 
0.061565 
1.253806 
0.266222 
0.01904 
23.06004 
1.3125 
6.071526 

I. 992454 

II. 76121 
0.997793 
1.479037 


"F  Ratio"  "Prob>F" 

0.000  1.0000 

0.225  0.6370 

0.382  0.5383 

6.306  0.0143 

0.379  0.5402 

0.229  0.6335 

0.546  0.4625 

0.001  0.9794 

0.030  0.8631 

1.228  0.2716 

0.138  0.7116 

0.002  0.9649 

0.503  0.4805 

0.433  0.5124 

0.900  0.3461 

0.138  0.7116 

2.741  0.1021 

3.121  0.0815 

1.567  0.2146 

0.063  0.8028 

0.303  0.5835 

0.208  0.6497 

0.001  0.9741 

0.115  0.7357 

0.405  0.5266 

0.653  0.4217 

0.033  0.8568 

0.674  0.4143 

0.142  0.7073 

0.010  0.9201 

12.458  0.0007 

0.706  0.4035 

3.280  0.0743 

1.078  0.3028 

6.354  0.0139 

0.536  0.4667 

0.797  0.3751 


Action 

Entered 

Entered 

Entered 

Entered 

Entered 

Entered 

Entered 


"Sig  Prob"  Seq  SS 

0.0242  12.1875 

0.0088  15.42722 

0.0164  12.07871 

0.0579  7.187571 

0.1181  4.748486 

0.1297  4.380843 

0.2146  2.901482 


RSquare  Cp  p 

0.0634  3.0312  2 

0.1437  -1.742  3 

0.2065  -5.046  4 

0.2439  -6.201  5 

0.2686  -6.286  6 

0.2914  -6.21  7 

0.3065  -5.484  8 


Appendix  C 


Response  Column  59 
Actual  by  Predicted  Plot 


1  2  3  4  5  6  7 

Column  59  Predicted  P=0.0003  RSq=0.31 
RMSE=1.3605 


Summary  of  Fit 


RSquare  0.306533 

RSquare  Adj  0.2391 13 

Root  Mean  Square  Error  1 .360533 

Mean  of  Response  4.1875 

Observations  (or  Sum  Wgts)  80 


Analysis  of  Variance 

Source  DF 

Model  7 

Error  72 

C.  Total  79 

Lack  Of  Fit 


Source  DF 

Sum  of  Squares 

Mean  Square 

F  Ratio 

Lack  Of  Fit  14 

24.99235 

1.78517 

0.9562 

Pure  Error  58 

108.28333 

1.86695 

Prob  >  F 

Total  Error  72 

133.27569 

0.5075 

Max  RSq 

0.4366 

Parameter  Estimates 

Term 

Estimate 

Std  Error 

t  Ratio  Prob>|t| 

Intercept 

4.5174278 

0.23955 

18.86  <.0001 

whattypesystemscommunications[1-0] 

-1.006132 

0.400667 

-2.51  0.0143 

whattypeenterpriseOES[1  -0] 

-1.639474 

0.990238 

-1.66  0.1021 

whattypeenterprisescripting[1  -0] 

-1.265895 

0.716529 

-1.77  0.0815 

whattypeenterpriseweb[1  -0] 

-0.534649 

0.427039 

-1.25  0.2146 

whatprocessCM[1  -0] 

-1.386206 

0.392742 

-3.53  0.0007 

whatprocessSWEngSuppt[1  -0] 

0.6302987 

0.348022 

1.81  0.0743 

whatproductsCOTS[1-0] 

1.0218048 

0.40537 

2.52  0.0139 

Effect  Tests 

Source 

Nparm 

DF 

Sum  of  Squares 

F  Ratio 

Prob  >  F 

whattypesystemscommunications 

1 

1 

11.672429 

6.3058 

0.0143 

whattypeenterpriseOES 

1 

1 

5.073979 

2.7411 

0.1021 

whattypeenterprisescripting 

1 

1 

5.777588 

3.1212 

0.0815 

whattypeenterpriseweb 

1 

1 

2.901482 

1 .5675 

0.2146 

whatprocessCM 

1 

1 

23.060036 

12.4578 

0.0007 

whatprocessSWEngSuppt 

1 

1 

6.071526 

3.2800 

0.0743 

wh  atprod  uctsCOT  S 

1 

1 

11.761207 

6.3538 

0.0139 

Risk  =  4.52  +  sys-comm(-I.OI)  +  ent-OES(-1.64)  +  ent-script(-1.27)  +  ent-web(-0.53)  +  proc-CM(-1 .39)  +  proc- 
SWEngSup(0.63)  +  prod-COTS(1 .02) 


Sum  of  Squares 
58.91181 
133.27569 
192.18750 


Mean  Square 
8.41597 
1.85105 


F  Ratio 
4.5466 
Prob  >  F 
0.0003 
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Stepwise  Fit  -  Old  Survey  Data  -  Consequences  (Quality) 

Response: 

Column  59 


Stepwise  Regression  Control 

Prob  to  Enter  0.250 

Prob  to  Leave  0.250 

Direction: 

Rules: 


Current  Estimates 

SSE  DFE 

138.55831  75 


MSE  RSquare 

1.8474441  0.1918 


RSquare  Adj 
0.1595 


Cp  AIC 

-16.09204  52.38563 


Lock 

X 


X  Intercept 

whattypesystemsavionics{0-1} 
whattypesystemsembedded{0-1 } 
whattypesystemscommunications{0-1} 
whattypesystemsdevice{0-1} 
whattypeshrinkbusiness{0-1} 
whattypeshrinkutilities{0-1 } 
whattypeshrinkinternet{0-1 } 
whattypecomponentdomain{0-1} 
whattypecomponentCASE{0-1} 
whattypecomponentclass{0- 1 } 
whattypecomponentOS{0-1} 
whattypecomponentdevelopment{0-1} 
whattypeenterpriseacctng{0-1 } 
whattypeenterprisemanufact{0-1} 
whattypeenterprisepayroll{0-1 } 
whattypeenterpriseOES{0- 1 } 
whattypeenterprisescripting{0-1} 
whattypeenterpriseweb{0-1 } 
whatprocessrequirements{0-1} 
whatprocessdesign{0- 1 } 
whatprocesstesting{0- 1 } 
whatprocessmaintenance{0-1} 

X  whatprocessreengineering{0-1} 

whatprocessappsuppt{0-1 } 
whatprocesstraining{0-1} 
whatprocessspecification{0-1 } 
whatprocessdocumentation{0-1 } 
whatprocesscoding{0- 1 } 
whatprocessfielding{0-1} 
whatprocessCM{0-1} 
whatprocesstoolsuppt{0-1} 

X  whatprocessSWEngSuppt{0-1} 

whatproductscustom{0-1 } 

X  whatproductsCOTS{0-1} 

whatproductscommoncust{0-1} 
whatproductsnone{0-1 } 


Step  History 

Step  Parameter  Action 

1  whatproductsCOTS{0-1}  Entered 

2  whatprocessreengineering{0-1}  Entered 

3  whatprocessSWEngSuppt{0-1}  Entered 


Estimate  nDF 

4.20996284  1 

0  1 

0  1 

0  1 

0  1 

0  1 

0  1 

0  1 

0  1 

0  1 

0  1 

0  1 

0  1 

0  1 

0  1 

0  1 

0  1 

0  1 

0  1 

0  1 

0  1 

0  1 

0  1 

-0.5762659  1 

0  1 

0  1 

0  1 

0  1 

0  1 

0  1 

0  1 

0  1 

0.31679381  1 

0  1 

0.6043975  1 

0  1 

0  1 


"Sig  Prob" 
0.0062 
0.0215 
0.0668 


Seq  SS 
15.98366 
10.50866 
6.392409 


SS 

0 

0.804632 

0.072328 

1.249862 

0.428972 

1.318692 

0.521273 

0.617671 

0.823718 

0.331022 

0.006299 

2.120124 

0.037316 

1.390717 

1.228873 

0.006299 

0.993111 

0.867781 

0.095182 

0.019064 

0.046973 

0.004928 

0.30243 

14.18207 

3.399e-8 

0.533073 

0.240311 

0.827112 

0.202728 

2.288202 

0.105642 

1.037728 

6.392409 

0.101684 

17.4637 

2.188209 

0.414216 


RSquare 

0.0932 

0.1545 

0.1918 


"F  Ratio"  "Prob>F" 

0.000  1.0000 

0.432  0.5129 

0.039  0.8447 

0.674  0.4144 

0.230  0.6331 

0.711  0.4018 

0.279  0.5986 

0.331  0.5666 

0.443  0.5080 

0.177  0.6750 

0.003  0.9539 

1.150  0.2871 

0.020  0.8881 

0.750  0.3892 

0.662  0.4184 

0.003  0.9539 

0.534  0.4671 

0.466  0.4968 

0.051  0.8222 

0.010  0.9199 

0.025  0.8746 

0.003  0.9592 

0.162  0.6886 

7.677  0.0070 

0.000  0.9999 

0.286  0.5945 

0.129  0.7209 

0.444  0.5071 

0.108  0.7429 

1.243  0.2686 

0.056  0.8128 

0.558  0.4573 

3.460  0.0668 

0.054  0.8163 

9.453  0.0029 

1.187  0.2794 

0.222  0.6390 


Cp  p 

-13.39  2 

-15.56  3 

-16.09  4 


Appendix  C 


Response  Column  59 
Actual  by  Predicted  Plot 


1  2  3  4  5  6  7 

Column  59  Predicted  P=0.0011  RSq=0.19 
RMSE=1.3592 


Summary  of  Fit 


RSquare  0.191811 

RSquare  Adj  0.159484 

Root  Mean  Square  Error  1 .359207 

Mean  of  Response  4.3291 14 

Observations  (or  Sum  Wgts)  79 


Analysis  of  Variance 

Source  DF 

Model  3 

Error  75 

C.  Total  78 

Lack  Of  Fit 

Source 
Lack  Of  Fit 
Pure  Error 
Total  Error 

Max  RSq 
0.2189 


DF 


Sum  of  Squares 
4  4.63583 

71  133.92247 

75  138.55831 


Mean  Square  F  Ratio 

1.15896  0.6144 

1.88623  Prob  >  F 

0.6536 


Sum  of  Squares 
32.88473 
138.55831 
171.44304 


Mean  Square 
10.9616 
1.8474 


F  Ratio 
5.9334 
Prob  >  F 
0.0011 


Parameter  Estimates 

Term 

Intercept 

whatprocessreengineering[1  -0] 
whatprocessSWEngSuppt[1  -0] 
whatproductsCOTS[1-0] 


Estimate 

4.5548882 

1.1525318 

-0.633588 

-1.208795 


Std  Error 
0.20017 
0.415977 
0.340612 
0.393161 


t  Ratio 
22.76 
2.77 
-1.86 
-3.07 


Prob>|t| 

<.0001 

0.0070 

0.0668 

0.0029 


Effect  T ests 

Source 

whatprocessreengineering 

whatprocessSWEngSuppt 

whatproductsCOTS 


Nparm  DF 

1  1 

1  1 

1  1 


Sum  of  Squares 
14.182067 
6.392409 
17.463697 


F  Ratio  Prob  >  F 

7.6766  0.0070 

3.4601  0.0668 

9.4529  0.0029 


Quality  =  4.55  +  proc-reeng(1.15)  +  proc-SWEngSup(-0.63)  +  prod-COTS(-1.21) 
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Stepwise  Fit  -  Old  Survey  Data  -  Consequences  (Rework) 

Response: 

Column  59 


Stepwise  Regression  Control 

Prob  to  Enter  0.250 

Prob  to  Leave  0.250 

Direction: 

Rules: 


Current  Estimates 

SSE  DFE 

100.79083  70 


MSE  RSquare 

1.439869  0.2441 


RSquare  Adj 
0.1686 


Cp  AIC 

-12.48343  35.99441 


Lock 

X 


X  Intercept 

whattypesystemsavionics{0-1} 
whattypesystemsembedded{0-1 } 

X  whattypesystemscommunications{0-1} 

X  whattypesystemsdevice{0-1} 

whattypeshrinkbusiness{0-1} 
whattypeshrinkutilities{0-1 } 

X  whattypeshrinkinternet{0-1} 

whattypecomponentdomain{0-1} 
whattypecomponentCASE{0-1} 
whattypecomponentclass{0- 1 } 
whattypecomponentOS{0-1} 
whattypecomponentdevelopment{0-1} 
whattypeenterpriseacctng{0-1 } 
whattypeenterprisemanufact{0-1 } 
whattypeenterprisepayroll{0-1 } 
whattypeenterpriseOES{0- 1 } 

X  whattypeenterprisescripting{0-1} 

whattypeenterpriseweb{0-1 } 
whatprocessrequirements{0-1} 
whatprocessdesign{0- 1 } 
whatprocesstesting{0- 1 } 
whatprocessmaintenance{0-1} 

X  whatprocessreengineering{0-1) 

whatprocessappsuppt{0-1 } 
whatprocesstraining{0-1} 
whatprocessspecification{0-1 } 
whatprocessdocumentation{0-1 } 
whatprocesscoding{0- 1 } 
whatprocessfielding{0-1} 
whatprocessCM{0-1} 
whatprocesstoolsuppt{0-1} 

X  whatprocessSWEngSuppt{0-1} 

whatproductscustom{0-1 } 

X  whatproductsCOTS{0-1} 

whatproductscommoncust{0-1} 
whatproductsnone{0-1 } 


Step  History 

Step  Parameter 

1  whatproductsCOTS{0-1} 

2  whattypecomponentdomain{0-1} 

3  whattypesystemsdevice{0-1} 

4  whatprocessreengineering{0-1} 

5  whattypeshrinkinternet{0-1} 

6  whattypecomponentdomain{0-1} 

7  whatprocessSWEngSuppt{0-1} 

8  whattypeenterprisescripting{0-1} 

9  whattypesystemscommunications{0-1 } 


Estimate  nDF 

4.97318561  1 

0  1 

0  1 

0.21887821  1 

-0.6822697  1 

0  1 

0  1 

-0.5654568  1 

0  1 

0  1 

0  1 

0  1 

0  1 

0  1 

0  1 

0  1 

0  1 

0.40105105  1 

0  1 

0  1 

0  1 

0  1 

0  1 

0.51804707  1 

0  1 

0  1 

0  1 

0  1 

0  1 

0  1 

0  1 

0  1 

-0.1955687  1 

0  1 

-0.5974604  1 

0  1 

0  1 


SS 

0 

0.038871 

0.075269 

2.219017 

7.071239 

0.000786 

0.159745 

5.173431 

1.819588 

0.344054 

0.689274 

1.363959 

0.940153 

0.292859 

0.22654 

0.689274 

1.213401 

2.234162 

0.71698 

0.01982 

0.171311 

0.90994 

0.023267 

9.241571 

0.000642 

0.002987 

0.238758 

0.301258 

1.877211 

0.053837 

1.288677 

0.283252 

2.241367 

0.534378 

16.34597 

0.19777 

0.010501 


"F  Ratio" 
0.000 
0.027 
0.052 
1.541 
4.911 
0.001 
0.110 
3.593 
1.269 
0.236 
0.475 
0.947 
0.650 
0.201 
0.155 
0.475 
0.841 
1.552 
0.494 
0.014 
0.117 
0.629 
0.016 
6.418 
0.000 
0.002 
0.164 
0.207 
1.310 
0.037 
0.894 
0.194 
1.557 
0.368 
11.352 
0.136 
0.007 


'Prob>F" 

1.0000 

0.8709 

0.8210 

0.2186 

0.0299 

0.9816 

0.7417 

0.0622 

0.2639 

0.6284 

0.4930 

0.3340 

0.4230 

0.6553 

0.6946 

0.4930 

0.3624 

0.2170 

0.4844 

0.9076 

0.7328 

0.4306 

0.8999 

0.0135 

0.9833 

0.9641 

0.6869 

0.6507 

0.2564 

0.8483 

0.3478 

0.6606 

0.2163 

0.5462 

0.0012 

0.7138 

0.9327 


Action 

Entered 

Entered 

Entered 

Entered 

Entered 

Removed 

Entered 

Entered 

Entered 


"Sig  Prob"  Seq  SS 

0.0040  13.86679 

0.1330  3.565351 

0.1428  3.337927 

0.1807  2.747747 

0.1712  2.8385 

0.2548  1.957808 

0.1178  3.663619 

0.2147  2.274184 

0.2186  2.219017 


RSquare  Cp  p 

0.1040  -15.3  2 

0.1307  -15.05  3 

0.1558  -14.69  4 

0.1764  -14.04  5 

0.1977  -13.44  6 

0.1830  -14.48  5 

0.2104  -14.28  6 

0.2275  -13.39  7 

0.2441  -12.48  8 


Appendix  C 


Response  Column  59 
Actual  by  Predicted  Plot 


Column  59  Predicted  P=0.0050  RSq=0.24 
RMSE=1.1999 


Summary  of  Fit 


RSquare  0.244141 

RSquare  Adj  0.168556 

Root  Mean  Square  Error  1.199945 

Mean  of  Response  4.269231 

Observations  (or  Sum  Wgts)  78 


Analysis  of  Variance 

Source 

DF 

Sum  of  Squares 

Mean  Square 

Model 

7 

32.55532 

4.65076 

Error 

70 

100.79083 

1 .43987 

C.  Total 

77 

133.34615 

F  Ratio 
3.2300 
Prob  >  F 
0.0050 


Lack  Of  Fit 

Source 
Lack  Of  Fit 
Pure  Error 
Total  Error 


DF 

13 

57 

70 


Sum  of  Squares  Mean  Square 

21.29083  1.63776 

79.50000  1.39474 

100.79083 


F  Ratio 
1.1742 
Prob  >  F 
0.3214 
Max  RSq 
0.4038 


Parameter  Estimates 


Term 

Estimate 

Std  Error 

t  Ratio  Prob>|t| 

Intercept 

4.0704064 

0.200654 

20.29  <.0001 

whattypesystemscommunications[1-0] 

-0.437756 

0.352626 

-1.24  0.2186 

whattypesystemsdevice[1  -0] 

1.3645393 

0.615743 

2.22  0.0299 

whattypeshrinkinternet[1  -0] 

1.1309135 

0.596625 

1.90  0.0622 

whattypeenterprisescripting[1-0] 

-0.802102 

0.643923 

-1.25  0.2170 

whatprocessreengineering[1  -0] 

-1.036094 

0.408967 

-2.53  0.0135 

whatprocessSWEngSuppt[1  -0] 

0.3911375 

0.313498 

1.25  0.2163 

whatproductsCOTS[1-0] 

1.1949208 

0.354646 

3.37  0.0012 

Effect  Tests 

Source 

Nparm  DF 

Sum  of  Squares 

F  Ratio 

Prob  >  F 

whattypesystemscommunications 

1  1 

2.219017 

1.5411 

0.2186 

whattypesystemsdevice 

1  1 

7.071239 

4.9110 

0.0299 

whattypeshrinkinternet 

1  1 

5.173431 

3.5930 

0.0622 

whattypeenterprisescripting 

1  1 

2.234162 

1.5516 

0.2170 

whatprocessreengineering 

1  1 

9.241571 

6.4183 

0.0135 

whatprocessSWEngSuppt 

1  1 

2.241367 

1.5566 

0.2163 

whatproductsCOTS 

1  1 

16.345966 

11.3524 

0.0012 

Rework  =  4.07  +  sys-comm(-0.44)  +  sys-dev(1.36)  +  shrink-int(1 . 1 3)  +  ent-script(-0.80)  +  proc-reeng(-1.04)  +  proc- 
SWEngSup(0.39)  +  prod-COTS(1.19) 
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Stepwise  Fit  -  Old  Survey  Data  -  Consequences  (Visibility) 

Response: 

Column  59 


Stepwise  Regression  Control 

Prob  to  Enter  0.250 

Prob  to  Leave  0.250 

Direction: 

Rules: 


Current  Estimates 

SSE  DFE 

80.322548  67 


MSE  RSquare 

1.198844  0.4229 


RSquare  Adj 
0.3196 


Cp 

-0.634968 


AIC 

26.3219 


Entered  Parameter 
X  Intercept 

whattypesystemsavionics{0-1} 
whattypesystemsembedded{0-1 } 

X  whattypesystemscommunications{0-1 } 

whattypesystemsdevice{0-1} 
whattypeshrinkbusiness{0-1} 
whattypeshrinkutilities{0-1 } 
whattypeshrinkinternet{0-1 } 
whattypecomponentdomain{0-1} 
whattypecomponentCASE{0-1 } 
whattypecomponentclass{0- 1 } 
whattypecomponentOS{0-1 } 

X  whattypecomponentdevelopment{0-1} 

whattypeenterpriseacctng{0-1} 
whattypeenterprisemanufact{0-1 } 
whattypeenterprisepayroll{0-1} 

X  whattypeenterpriseOES{0-1} 

X  whattypeenterprisescripting{0-1} 

X  whattypeenterpriseweb{0-1} 

whatprocessrequirements{0-1} 

X  whatprocessdesign{0-1} 

whatprocesstesting{0-1} 

X  whatprocessmaintenance{0-1} 

X  whatprocessreengineering{0-1} 

whatprocessappsuppt{0-1 } 
whatprocesstraining{0-1} 
whatprocessspecification{0-1 } 
whatprocessdocumentation{0-1 } 
whatprocesscoding{0- 1 } 

X  whatprocessfielding{0-1} 

whatprocessCM{0-1} 
whatprocesstoolsuppt{0-1} 

X  whatprocessSWEngSuppt{0-1} 

X  whatproductscustom{0-1} 

whatproductsCOTS{0-1} 
whatproductscommoncust{0-1} 
whatproductsnone{0-1 } 


Step  History 

Step  Parameter 


10 

11 

12 

13 

14 


whatprocessreengineering{0-1 } 

whatproductsnone{0-1 } 

whatproductscustom{0-1 } 

whattypeenterpriseOES{0- 1 } 

whatprocessfielding{0-1} 

whattypeenterprisescripting{0-1} 

whattypeshrinkinternet{0-1} 

whattypesystemscommunications{0-1} 

whatprocessmaintenance{0-1} 

whattypecomponentdevelopment{0-1} 

whatprocessdesign{0-1} 

whattypeshrinkinternet{0-1} 

whatprocessSWEngSuppt{0-1 } 

whattypeenterpriseweb{0-1 } 


Estimate  nDF 

5.98677311  1 

0  1 

0  1 

-0.3455468  1 

0  1 

0  1 

0  1 

0  1 

0  1 

0  1 

0  1 

0  1 

0.356298  1 

0  1 

0  1 

0  1 

-0.9907837  1 

-0.6244449  1 

0.24081165  1 

0  1 

0.27918704  1 

0  1 

-0.338678  1 

-0.7537735  1 

0  1 

0  1 

0  1 

0  1 

0  1 

0.4390628  1 

0  1 

0  1 

0.19014744  1 

-0.4082429  1 

0  1 

0  1 

-1.2483768  1 


SS 

0 

0.518286 

0.602672 

5.183555 

0.001077 

0.213281 

0.367963 

0.299066 

0.701401 

0.438187 

0.094628 

0.852281 

4.779899 

0.476471 

0.151067 

0.094628 

6.935252 

5.375618 

I . 688966 
0.172193 
3.685938 
0.320998 
6.758802 
17.07158 

0.50352 

0.231205 

0.329009 

0.008766 

0.347492 

6.55602 

0.201871 

0.002945 

2.001007 

8.168385 

0.036239 

0.981731 

I I . 04777 


"F  Ratio" 
0.000 
0.429 
0.499 
4.324 
0.001 
0.176 
0.304 
0.247 
0.581 
0.362 
0.078 
0.708 
3.987 
0.394 
0.124 
0.078 
5.785 
4.484 
1.409 
0.142 
3.075 
0.265 
5.638 
14.240 
0.416 
0.191 
0.271 
0.007 
0.287 
5.469 
0.166 
0.002 
1.669 
6.814 
0.030 
0.817 
9.215 


'Prob>F" 

1.0000 

0.5149 

0.4824 

0.0414 

0.9764 

0.6764 

0.5834 

0.6211 

0.4485 

0.5494 

0.7811 

0.4032 

0.0499 

0.5324 

0.7255 

0.7811 

0.0189 

0.0379 

0.2394 

0.7077 

0.0841 

0.6085 

0.0204 

0.0003 

0.5210 

0.6639 

0.6041 

0.9326 

0.5941 

0.0224 

0.6847 

0.9609 

0.2008 

0.0112 

0.8635 

0.3694 

0.0034 


Action 

Entered 

Entered 

Entered 

Entered 

Entered 

Entered 

Entered 

Entered 

Entered 

Entered 

Entered 

Removed 

Entered 

Entered 


"Sig  Prob"  Seq  SS 

0.0044  13.80005 

0.0324  7.280303 

0.0445  6.148685 

0.0698  4.834524 

0.0563  5.179103 

0.0720  4.449566 

0.1234  3.183774 

0.1257  3.085155 

0.1318  2.932973 

0.1419  2.737541 

0.1373  2.752772 

0.3797  0.951842 

0.2334  1.743379 

0.2394  1 .688966 


RSquare  Cp  p 

0.0991  7.3054  2 

0.1515  4.4685  3 

0.1956  2.3834  4 

0.2304  1.1714  5 

0.2676  -0.269  6 

0.2995  -1 .226  7 

0.3224  -1.341  8 

0.3446  -1.391  9 

0.3657  -1.339  10 

0.3853  -1.158  11 

0.4051  -0.987  12 

0.3983  -2.355  1 1 

0.4108  -1.513  12 

0.4229  -0.635  13 
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Response  Column  59 
Actual  by  Predicted  Plot 


I 


0  1  2  3  4  5  6  7 

Column  59  Predicted  P<.0001  RSq=0.42 
RMSE=1 .0949 

Summary  of  Fit 


RSquare  0.422918 

RSquare  Adj  0.31956 

Root  Mean  Square  Error  1.094917 

Mean  of  Response  3.3125 

Observations  (or  Sum  Wgts)  80 

Analysis  of  Variance 

Source  DF  Sum  of  Squares  Mean  Square 

Model  12  58.86495  4.90541 

Error  67  80.32255  1.19884 

C.  Total  79  139.18750 

Lack  Of  Fit 


Source  DF 

Lack  Of  Fit  35 

Pure  Error  32 

Total  Error  67 


Parameter  Estimates 

Term 

Intercept 

whattypesystemscommunications[1-0] 
whattypecomponentdevelopment[1-0] 
whattypeenterpriseOES[1  -0] 
whattypeenterprisescripting[1-0] 
whattypeenterpriseweb[1  -0] 
whatprocessdesign[1-0] 
whatprocessmaintenance[1  -0] 
whatprocessreengineering[1-0] 
whatprocessfieldingjl  -0] 
whatprocessSWEngSuppt[1-0] 
whatproductscustom[1  -0] 
whatproductsnone[1  -0] 

Effect  Tests 

Source 

whattypesystemscommunications 

whattypecomponentdevelopment 

whattypeenterpriseOES 

whattypeenterprisescripting 

whattypeenterpriseweb 

whatprocessdesign 

whatprocessmaintenance 

whatprocessreengineering 

whatprocessfielding 

whatprocessSWEngSuppt 

whatproductscustom 

whatproductsnone 


of  Squares  Mean  Square 

41.289214  1.17969 

39.033333  1.21979 

80.322548 


Estimate 

2.7824335 

0.6910937 

-0.712596 

1.9815674 

1.2488897 

-0.481623 

-0.558374 

0.677356 

1.507547 

-0.878126 

-0.380295 

0.8164858 

2.4967535 


Nparm  DF 

1  1 

1  1 

1  1 

1  1 

1  1 

1  1 

1  1 

1  1 

1  1 

1  1 

1  1 

1  1 


F  Ratio 
4.0918 
Prob  >  F 
<.0001 


F  Ratio 
0.9671 
Prob  >  F 
0.5403 
Max  RSq 
0.7196 


Std  Error 
0.321509 
0.332357 
0.356874 
0.823871 
0.589782 
0.405769 
0.318444 
0.285275 
0.399499 
0.375507 
0.294359 
0.312796 
0.822469 


t  Ratio  Prob>|t| 

8.65  <.0001 

2.08  0.0414 

-2.00  0.0499 

2.41  0.0189 

2.12  0.0379 

-1.19  0.2394 

-1.75  0.0841 

2.37  0.0204 

3.77  0.0003 

-2.34  0.0224 

-1.29  0.2008 

2.61  0.0112 

3.04  0.0034 


Sum  of  Squares 
5.183555 
4.779899 
6.935252 
5.375618 
1 .688966 
3.685938 
6.758802 
17.071582 
6.556020 
2.001007 
8.168385 
11.047775 


F  Ratio  Prob  >  F 

4.3238  0.0414 

3.9871  0.0499 

5.7849  0.0189 

4.4840  0.0379 

1 .4088  0.2394 

3.0746  0.0841 

5.6378  0.0204 

14.2400  0.0003 

5.4686  0.0224 

1.6691  0.2008 

6.8136  0.0112 

9.2154  0.0034 


Visibility  =  2.78  +  sys-comm(0.69)  +  comp-dev(-0.71)  +  ent-OES(1.98)  +  ent-script(1.25)  +  ent-web(-0.48)  +  proc-des(- 
0.56)  +  proc-maint(0.68)  +  proc-reeng(1.51)  +  roc-field(-0.88)  +  proc-SWEngSup(-0.38)  +  prod-cust(0.82)  +  prod- 
none(2.50) 
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Stepwise  Fit  -  Old  Survey  Data  -  Consequences  (ControlProduct) 

Response: 

Column  59 


Stepwise  Regression  Control 


Prob  to  Enter  0.250 

Prob  to  Leave  0.250 

Direction: 

Rules: 


Current  Estimates 

SSE  DFE 

82.519081  69 


MSE  RSquare 

1.1959287  0.3860 


RSquare  Adj 
0.2970 


Cp  AIC 

-2.38735  24.48023 


Lock 

X 


X  Intercept 

whattypesystemsavionics{0-1} 
whattypesystemsembedded{0-1 } 
whattypesystemscommunications{0-1} 
X  whattypesystemsdevice{0-1} 

whattypeshrinkbusiness{0-1} 

X  whattypeshrinkutilities{0-1} 

whattypeshrinkinternet{0-1 } 

X  whattypecomponentdomain{0-1} 

whattypecomponentCASE{0-1} 
whattypecomponentclass{0- 1 } 
whattypecomponentOS{0-1 } 
whattypecomponentdevelopment{0-1} 
whattypeenterpriseacctng{0-1 } 
whattypeenterprisemanufact{0-1 } 
whattypeenterprisepayroll{0-1 } 
whattypeenterpriseOES{0- 1 } 
whattypeenterprisescripting{0-1} 
whattypeenterpriseweb{0-1 } 
whatprocessrequirements{0-1} 
whatprocessdesign{0- 1 } 
whatprocesstesting{0-1} 
whatprocessmaintenance{0-1} 

X  whatprocessreengineering{0-1} 

X  whatprocessappsuppt{0-1} 

whatprocesstraining{0-1} 
whatprocessspecification{0-1 } 
whatprocessdocumentation{0-1 } 
whatprocesscoding{0- 1 } 

X  whatprocessfielding{0-1} 

whatprocessCM{0-1} 

X  whatprocesstoolsuppt{0-1} 

X  whatprocessSWEngSuppt{0-1} 

whatproductscustom{0-1 } 

X  whatproductsCOTS{0-1} 

X  whatproductscommoncust{0-1} 

whatproductsnone{0-1 } 


Step  History 

Step  Parameter 

1  whatproductscommoncust{0-1} 

2  whatprocessmaintenance{0-1} 

3  whatprocesstoolsuppt{0-1} 

4  whatprocessappsuppt{0-1} 

5  whatproductsCOTS{0-1} 

6  whatprocessfielding{0-1} 

7  whattypesystemsdevice{0-1} 

8  whatprocessSWEngSuppt{0-1} 

9  whattypeshrinkutilities{0-1} 

10  whattypecomponentdomain{0-1} 

11  whatprocessmaintenance{0-1} 

12  whatprocessreengineering{0-1} 


Action 

Entered 

Entered 

Entered 

Entered 

Entered 

Entered 

Entered 

Entered 

Entered 

Entered 

Removed 

Entered 


Estimate  nDF 

3.56042654  1 

0  1 

0  1 

0  1 

0.54209815  1 

0  1 

-0.8827025  1 

0  1 

0.42847915  1 

0  1 

0  1 

0  1 

0  1 

0  1 

0  1 

0  1 

0  1 

0  1 

0  1 

0  1 

0  1 

0  1 

0  1 

-0.3056548  1 

-0.9057526  1 

0  1 

0  1 

0  1 

0  1 

0.32812655  1 

0  1 

0.3812332  1 

0.2297158  1 

0  1 

0.4831796  1 

0.33076121  1 

0  1 


"Sig  Prob" 
0.0056 
0.0329 
0.0562 
0.0388 
0.0661 
0.1085 
0.1337 
0.1476 
0.1641 
0.1326 
0.3063 
0.1192 


Seq  SS 
12.6498 
7.033841 
5.408824 
6.088748 
4.635403 
3.440173 
2.946144 
2.701419 
2.457849 
2.826315 
1 .29648 
2.976387 


SS 

0 

0.030868 

0.175785 

0.037494 

4.280597 

0.149815 

4.723598 

0.005012 

2.908912 

0.14849 

0.001096 

0.908058 

0.574318 

1.540991 

0.263658 

0.001096 

0.304817 

0.918888 

0.783535 

0.79934 

0.198203 

0.20355 

0.860112 

2.976387 

13.38189 

0.3948 

0.452618 

0.987322 

0.150195 

3.220936 

0.11788 

4.993909 

2.205656 

0.413515 

8.525724 

4.96574 

1.388063 


RSquare 

0.0941 

0.1465 

0.1867 

0.2320 

0.2665 

0.2921 

0.3140 

0.3341 

0.3524 

0.3735 

0.3638 

0.3860 


"F  Ratio"  "Prob>F" 

0.000  1.0000 

0.025  0.8737 

0.145  0.7044 

0.031  0.8610 

3.579  0.0627 

0.124  0.7262 

3.950  0.0508 

0.004  0.9489 

2.432  0.1234 

0.123  0.7273 

0.001  0.9761 

0.757  0.3875 

0.477  0.4923 

1.294  0.2593 

0.218  0.6421 

0.001  0.9761 

0.252  0.6172 

0.766  0.3846 

0.652  0.4223 

0.665  0.4176 

0.164  0.6870 

0.168  0.6830 

0.716  0.4003 

2.489  0.1192 

11.190  0.0013 

0.327  0.5694 

0.375  0.5423 

0.823  0.3674 

0.124  0.7258 

2.693  0.1053 

0.097  0.7561 

4.176  0.0448 

1.844  0.1789 

0.342  0.5603 

7.129  0.0095 

4.152  0.0454 

1.163  0.2846 


Cp  p 

6.0435  2 

3.3032  3 

1.658  4 

-0.445  5 

-1.569  6 

-1.888  7 

-1.873  8 

-1.694  9 

-1.35  10 

-1 .255  1 1 

-2.381  10 

-2.387  1 1 


Appendix  C 


Response  Column  59 
Actual  by  Predicted  Plot 


Column  59  Predicted  P=0.0001  RSq=0.39 
RMSE=1.0936 


Summary  of  Fit 


RSquare 
RSquare  Adj 
Root  Mean  Square  Error 
Mean  of  Response 
Observations  (or  Sum  Wgts) 

Analysis  of  Variance 


Source  DF 

Model  10 

Error  69 

C.  Total  79 


0.385962 

0.296971 

1.093585 

3.7125 

80 


Sum  of  Squares 
51.86842 
82.51908 
134.38750 


Mean  Square  F  Ratio 

5.18684  4.3371 

1.19593  Prob  >  F 

0.0001 


Lack  Of  Fit 


Source  DF 

Lack  Of  Fit  21 

Pure  Error  48 

Total  Error  69 


Sum  of  Squares 
30.635747 
51.883333 
82.519081 


Mean  Square  F  Ratio 

1.45885  1.3497 

1.08090  Prob  >  F 

0.1929 
Max  RSq 
0.6139 


Parameter  Estimates 


Term 

Estimate 

Std  Error 

t  Ratio  Prob>|t| 

Intercept 

4.1899103 

0.180433 

23.22  <.0001 

whattypesystemsdevice[1  -0] 

-1.084196 

0.573071 

-1.89  0.0627 

whattypeshrinkutilities[1-0] 

1.765405 

0.888302 

1.99  0.0508 

whattypecomponentdomain[1  -0] 

-0.856958 

0.549474 

-1.56  0.1234 

whatprocessreengineering[1-0] 

0.6113096 

0.387498 

1.58  0.1192 

whatprocessappsuppt[1  -0] 

1.8115051 

0.541544 

3.35  0.0013 

Whatprocessfielding[1  -0] 

-0.656253 

0.399883 

-1.64  0.1053 

whatprocesstoolsuppt[1  -0] 

-0.762466 

0.373124 

-2.04  0.0448 

whatprocessSWEngSuppt[1  -0] 

-0.459432 

0.338302 

-1.36  0.1789 

whatproductsCOTSjl  -0] 

-0.966359 

0.361931 

-2.67  0.0095 

whatproductscommoncust[1  -0] 

-0.661522 

0.324642 

-2.04  0.0454 

Effect  Tests 

Source 

Nparm  DF 

Sum  of  Squares 

F  Ratio 

Prob  >  F 

whattypesystemsdevice 

1  1 

4.280597 

3.5793 

0.0627 

whattypeshrinkutilities 

1  1 

4.723598 

3.9497 

0.0508 

whattypecomponentdomain 

1  1 

2.908912 

2.4323 

0.1234 

whatprocessreengineering 

1  1 

2.976387 

2.4888 

0.1192 

whatprocessappsuppt 

1  1 

13.381887 

11.1895 

0.0013 

Whatprocessfielding 

1  1 

3.220936 

2.6933 

0.1053 

whatprocesstoolsuppt 

1  1 

4.993909 

4.1758 

0.0448 

whatprocessSWEngSuppt 

1  1 

2.205656 

1 .8443 

0.1789 

whatproductsCOTS 

1  1 

8.525724 

7.1290 

0.0095 

whatproductscommoncust 

1  1 

4.965740 

4.1522 

0.0454 

ControlProduct  =  4.19  +  sys-dev(-1.08)  +  shrink-util(1 .77)  +  comp-domain(-0.86)  +  proc-reeng(0.61)  +  proc-appsup(1.81) 
+  proc-field(-0.66)  +  proc-toolsup(-0.76)  +  proc-SWEngSup(-0.46)  +  prod-COTS(-0.97)  +  prod-comcust(-0.66) 


Page  14 


Stepwise  Fit  -  Old  Survey  Data  -  Consequences  (ChangeCost) 

Response: 

Column  59 


Stepwise  Regression  Control 

Prob  to  Enter  0.250 

Prob  to  Leave  0.250 

Direction: 

Rules: 


Current  Estimates 

SSE  DFE 

80.997432  69 


MSE  RSquare 

1.1738758  0.3453 


RSquare  Adj  Cp 

0.2599  -7.160605 


AIC 

21.9726 


Lock 

X 


X  Intercept 

X  whattypesystemsavionics{0-1} 

whattypesystemsembedded{0-1 } 
whattypesystemscommunications{0-1} 
whattypesystemsdevice{0-1} 
whattypeshrinkbusiness{0-1} 
whattypeshrinkutilities{0-1 } 

X  whattypeshrinkinternet{0-1} 

whattypecomponentdomain{0-1} 
whattypecomponentCASE{0-1} 
whattypecomponentclass{0- 1 } 

X  whattypecomponentOS{0-1} 

whattypecomponentdevelopment{0-1} 
X  whattypeenterpriseacctng{0-1} 

X  whattypeenterprisemanufact{0-1} 

whattypeenterprisepayroll{0-1 } 

X  whattypeenterpriseOES{0-1} 

whattypeenterprisescripting{0-1} 

X  whattypeenterpriseweb{0-1} 

whatprocessrequirements{0-1} 
whatprocessdesign{0- 1 } 
whatprocesstesting{0- 1 } 
whatprocessmaintenance{0-1} 
whatprocessreengineering{0-1} 
whatprocessappsuppt{0-1 } 
whatprocesstraining{0-1} 
whatprocessspecification{0-1 } 
whatprocessdocumentation{0-1 } 
whatprocesscoding{0- 1 } 
whatprocessfielding{0-1} 
whatprocessCM{0-1} 
whatprocesstoolsuppt{0-1} 

X  whatprocessSWEngSuppt{0-1} 

whatproductscustom{0-1 } 

X  whatproductsCOTS{0-1} 

whatproductscommoncust{0-1} 
whatproductsnone{0-1 } 


Step  History 

Step  Parameter 

1  whattypecomponentOS{0-1} 

2  whatproductsCOTS{0-1} 

3  whattypecomponentdomain{0-1} 

4  whattypeenterpriseacctng{0-1} 

5  whattypeenterpriseOES{0-1} 

6  whatprocesstoolsuppt{0-1} 

7  whattypeenterprisemanufact{0-1} 

8  whattypeshrinkinternet{0-1} 

9  whattypeenterpriseweb{0-1} 

10  whattypesystemsavionics{0-1} 

11  whattypecomponentdomain{0-1} 

12  whatprocessSWEngSuppt{0-1} 

13  whatprocesstoolsuppt{0-1} 


Action 

Entered 

Entered 

Entered 

Entered 

Entered 

Entered 

Entered 

Entered 

Entered 

Entered 

Removed 

Entered 

Removed 


Estimate  nDF 

6.56466218  1 

-0.4290094  1 

0  1 

0  1 

0  1 

0  1 

0  1 

-0.6955694  1 

0  1 

0  1 

0  1 

-0.801777  1 

0  1 

-0.6708692  1 

-0.7388432  1 

0  1 

0.98388367  1 

0  1 

0.44158039  1 

0  1 

0  1 

0  1 

0  1 

0  1 

0  1 

0  1 

0  1 

0  1 

0  1 

0  1 

0  1 

0  1 

-0.2254156  1 

0  1 

-0.3571599  1 

0  1 

0  1 


SS 

0 

3.160353 

0.049415 

1.113881 

0.030806 

0.456482 

0.553773 

6.987774 

0.578532 

0.017193 

0.167294 

10.96923 

0.088069 

7.416831 

2.056883 

0.167294 

6.844512 

0.467375 

5.728142 

0.020758 

0.477487 

0.264923 

0.031899 

0.1531 

0.158275 

0.039953 

0.013598 

0.939663 

0.011639 

0.574795 

1.457047 

0.016326 

3.136669 

0.219892 

5.603729 

0.310219 

0.69321 


"Sig  Prob" 
0.0467 
0.0511 
0.0675 
0.0577 
0.0317 
0.0926 
0.2020 
0.2150 
0.1017 
0.1811 
0.3457 
0.1771 
0.9071 


Seq  SS 
6.237735 
5.775309 
4.900137 
5.108657 
6.268299 
3.701944 
2.093894 
1.961383 
3.361312 
2.204849 
1.088921 
2.215816 
0.016326 


RSquare 

0.0504 

0.0971 

0.1367 

0.1780 

0.2287 

0.2586 

0.2755 

0.2914 

0.3185 

0.3363 

0.3275 

0.3455 

0.3453 


"F  Ratio" 
0.000 
2.692 
0.042 
0.948 
0.026 
0.385 
0.468 
5.953 
0.489 
0.014 
0.141 
9.344 
0.074 
6.318 
1.752 
0.141 
5.831 
0.395 
4.880 
0.017 
0.403 
0.223 
0.027 
0.129 
0.133 
0.034 
0.011 
0.798 
0.010 
0.486 
1.246 
0.014 
2.672 
0.185 
4.774 
0.261 
0.587 


Cp 

0.1911 

-1.505 

-2.641 

-3.911 

-5.923 

-6.292 

-5.632 

-4.887 

-5.039 

-4.45 

-5.753 

-5.171 

-7.161 


'Prob>F" 

1.0000 

0.1054 

0.8392 

0.3336 

0.8727 

0.5368 

0.4962 

0.0173 

0.4867 

0.9047 

0.7087 

0.0032 

0.7864 

0.0143 

0.1900 

0.7087 

0.0184 

0.5320 

0.0305 

0.8954 

0.5276 

0.6382 

0.8705 

0.7208 

0.7163 

0.8552 

0.9152 

0.3748 

0.9215 

0.4881 

0.2683 

0.9071 

0.1067 

0.6684 

0.0323 

0.6108 

0.4462 


P 

2 

3 

4 

5 

6 

7 

8 
9 

10 

11 

10 

11 

10 


Appendix  C 


Response  Column  59 
Actual  by  Predicted  Plot 


1  2  3  4  5  6  7 

Column  59  Predicted  P=0.0003  RSq=0.35 
RMSE=1.0835 


Summary  of  Fit 


RSquare  0.345325 

RSquare  Adj  0.259932 

Root  Mean  Square  Error  1 .083456 

Mean  of  Response  4.518987 

Observations  (or  Sum  Wgts)  79 


Analysis  of  Variance 


Source  DF 

Sum  of  Squares 

Mean  Square 

F  Ratio 

Model  9 

42.72409 

4.74712 

4.0440 

Error  69 

80.99743 

1.17388 

Prob  >  F 

C.  Total  78 

123.72152 

0.0003 

Lack  Of  Fit 

Source  DF 

Sum  of  Squares 

Mean  Square 

F  Ratio 

Lack  Of  Fit  9 

10.049922 

1.11666 

0.9444 

Pure  Error  60 

70.947510 

1.18246 

Prob  >  F 

Total  Error  69 

80.997432 

0.4943 
Max  RSq 

0.4266 

Parameter  Estimates 

Term 

Estimate 

Std  Error  1 

:  Ratio  Prob>|t| 

Intercept 

4.0714825 

0.18409 

22.12  <.0001 

whattypesystemsavionics[1  -0] 

0.8580188 

0.522926 

1.64  0.1054 

whattypeshrinkinternet[1  -0] 

1.3911389 

0.57018 

2.44  0.0173 

whattypecomponentOS[1  -0] 

1.603554 

0.524574 

3.06  0.0032 

whattypeenterpriseacctng[1  -0] 

1.3417385 

0.533789 

2.51  0.0143 

whattypeenterprisemanufact[1  -0] 

1 .4776863 

1.11632 

1.32  0.1900 

whattypeenterpriseOES[1  -0] 

-1.967767 

0.814917 

-2.41  0.0184 

whattypeenterpriseweb[1  -0] 

-0.883161 

0.399801 

-2.21  0.0305 

whatprocessSWEngSuppt[1  -0] 

0.450831 1 

0.275798 

1.63  0.1067 

whatproductsCOTS[1-0] 

0.7143198 

0.326938 

2.18  0.0323 

Effect  Tests 

Source 

Nparm 

DF  Sum  of  Squares 

F  Ratio 

Prob  >  F 

whattypesystemsavionics 

1 

1 

3.160353 

2.6922 

0.1054 

whattypeshrinkinternet 

1 

1 

6.987774 

5.9527 

0.0173 

whattypecomponentOS 

1 

1 

10.969235 

9.3445 

0.0032 

whattypeenterpriseacctng 

1 

1 

7.416831 

6.3182 

0.0143 

whattypeenterprisemanufact 

1 

1 

2.056883 

1.7522 

0.1900 

whattypeenterpriseOES 

1 

1 

6.844512 

5.8307 

0.0184 

whattypeenterpriseweb 

1 

1 

5.728142 

4.8797 

0.0305 

whatprocessSWEngSuppt 

1 

1 

3.136669 

2.6721 

0.1067 

wh  atprod  uctsCOT  S 

1 

1 

5.603729 

4.7737 

0.0323 

ChangeCost  =  4.07  +  sys-avia(0.86)  +  shrink-int(1.39)  +  comp-OS(1.60)  +  ent-acct(1.34)  +  ent-mnft(1.48)  +  ent-OES(- 
1.97)  +  ent-web(-0.88)  +  proc-SWEngSup(0.45)  +  prod-COTS(0.71) 
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Stepwise  Fit  -  Old  Survey  Data  -  Consequences  (LangCulture) 

Response: 

Column  59 

Stepwise  Regression  Control 

Prob  to  Enter  0.250 

Prob  to  Leave  0.250 

Direction: 

Rules: 


Current  Estimates 

SSE  DFE 

25.650194  66 


MSE  RSquare 

0.3886393  0.4599 


RSquare  Adj 
0.3698 


Cp  AIC 

-4.602552  -62.7483 


Lock 

X 


X  Intercept 

X  whattypesystemsavionics{0-1} 

whattypesystemsembedded{0-1 } 
whattypesystemscommunications{0-1} 
whattypesystemsdevice{0-1 } 
whattypeshrinkbusiness{0-1} 

X  whattypeshrinkutilities{0-1} 

whattypeshrinkinternet{0-1 } 
whattypecomponentdomain{0-1} 
whattypecomponentCASE{0-1 } 
whattypecomponentclass{0- 1 } 

X  whattypecomponentOS{0-1} 

whattypecomponentdevelopment{0-1} 
X  whattypeenterpriseacctng{0-1} 

whattypeenterprisemanufact{0-1 } 
whattypeenterprisepayroll{0-1 } 

X  whattypeenterpriseOES{0-1} 

whattypeenterprisescripting{0-1} 
whattypeenterpriseweb{0-1 } 

X  whatprocessrequirements{0-1} 

X  whatprocessdesign{0-1} 

whatprocesstesting{0-1} 
whatprocessmaintenance{0-1} 
whatprocessreengineering{0-1} 
whatprocessappsuppt{0-1 } 
whatprocesstraining{0-1} 
whatprocessspecification{0-1 } 
whatprocessdocumentation{0-1 } 
whatprocesscoding{0- 1 } 

X  whatprocessfielding{0-1} 

whatprocessCM{0-1} 
whatprocesstoolsuppt{0-1} 

X  whatprocessSWEngSuppt{0-1} 

X  whatproductscustom{0-1} 

whatproductsCOTS{0-1} 

X  whatproductscommoncust{0-1} 

whatproductsnone{0-1 } 


Step  History 

Step  Parameter 

1  whattypeshrinkutilities{0-1} 

2  whatprocessdesign{0-1} 

3  whattypeenterpriseacctng{0-1} 

4  whattypesystemsavionics{0-1} 

5  whatproductscommoncust{0-1} 

6  whatprocessfielding{0-1} 

7  whattypecomponentOS{0-1} 

8  whatprocessrequirements{0-1} 

9  whatprocessSWEngSuppt{0-1} 

10  whatproductscustom{0-1} 

11  whattypeenterpriseOES{0-1} 


Action 

Entered 

Entered 

Entered 

Entered 

Entered 

Entered 

Entered 

Entered 

Entered 

Entered 

Entered 


Estimate  nDF 

5.13982864  1 

-0.2610471  1 

0  1 

0  1 

0  1 

0  1 

-0.7499191  1 

0  1 

0  1 

0  1 

0  1 

0.17512387  1 

0  1 

-0.4041835  1 

0  1 

0  1 

0.30398637  1 

0  1 

0  1 

0.22733546  1 

-0.3613794  1 

0  1 

0  1 

0  1 

0  1 

0  1 

0  1 

0  1 

0  1 

0.29637258  1 

0  1 

0  1 

-0.2480822  1 

0.18625596  1 

0  1 

-0.2388453  1 

0  1 


"Sig  Prob" 
0.0050 
0.0558 
0.0545 
0.0646 
0.0320 
0.0145 
0.1075 
0.1212 
0.0414 
0.0491 
0.2101 


Seq  SS 
4.697706 
2.050847 
1.99847 
1 .782643 
2.301455 
2.816809 
1.16504 
1 .056626 
1.770243 
1 .574582 
0.622565 


SS 

0 

1.154913 

0.418222 

0.187069 

0.0602 

0.046931 

4.174295 

0.001682 

0.121123 

0.178824 

0.147903 

0.531482 

0.431405 

2.636964 

0.035927 

0.147903 

0.622565 

0.149021 

0.245273 

2.097215 

5.826266 

0.105452 

0.276994 

0.001122 

0.000081 

0.058085 

0.109741 

0.259248 

0.004854 

2.729608 

0.07496 

0.075144 

2.71569 

1.687473 

0.013608 

2.40028 

0.012225 


RSquare 

0.0989 

0.1421 

0.1842 

0.2217 

0.2702 

0.3295 

0.3541 

0.3763 

0.4136 

0.4467 

0.4599 


"F  Ratio" 
0.000 
2.972 
1.077 
0.478 
0.153 
0.119 
10.741 
0.004 
0.308 
0.456 
0.377 
1.368 
1.112 
6.785 
0.091 
0.377 
1.602 
0.380 
0.628 
5.396 
14.991 
0.268 
0.710 
0.003 
0.000 
0.148 
0.279 
0.664 
0.012 
7.024 
0.191 
0.191 
6.988 
4.342 
0.035 
6.176 
0.031 


Cp 

8.4045 

6.4549 

4.6063 

3.1732 

0.741 

-2.684 

-2.927 

-2.962 

-4.371 

-5.404 

-4.603 


'Prob>F" 

1.0000 

0.0894 

0.3031 

0.4920 

0.6970 

0.7311 

0.0017 

0.9482 

0.5806 

0.5017 

0.5414 

0.2464 

0.2956 

0.0113 

0.7637 

0.5414 

0.2101 

0.5398 

0.4311 

0.0233 

0.0003 

0.6062 

0.4027 

0.9576 

0.9886 

0.7022 

0.5990 

0.4182 

0.9120 

0.0101 

0.6639 

0.6636 

0.0102 

0.0411 

0.8532 

0.0155 

0.8608 


P 

2 

3 

4 

5 

6 

7 

8 
9 

10 

11 

12 


Appendix  C 


Response  Column  59 


Actual  by  Predicted  Plot 


I 


2  3  4  5  6  7 

Column  59  Predicted  P<.0001  RSq=0.46 
RMSE=0.6234 


Summary  of  Fit 


RSquare 
RSquare  Adj 
Root  Mean  Square  Error 
Mean  of  Response 
Observations  (or  Sum  Wgts) 
Analysis  of  Variance 
Source  DF 

Model  1 1 

Error  66 

C.  Total  77 

Lack  Of  Fit 

Source  DF 

Lack  Of  Fit  25 

Pure  Error  41 

Total  Error  66 


0.45985 

0.369825 

0.623409 

4.487179 

78 


Sum  of  Squares 
21.836986 
25.650194 
47.487179 


Mean  Square  F  Ratio 

1.98518  5.1080 

0.38864  Prob  >  F 

<.0001 


Sum  of  Squares 
14.804739 
10.845455 
25.650194 


Mean  Square  F  Ratio 

0.592190  2.2387 

0.264523  Prob  >  F 

0.0107 
Max  RSq 
0.7716 


Parameter  Estimates 

Term 

Intercept 

whattypesystemsavionics[1  -0] 
whattypeshrinkutilities[1  -0] 
whattypecomponentOS[1  -0] 
whattypeenterpriseacctngjl  -0] 
whattypeenterpriseOES[1  -0] 
whatprocessrequirements[1  -0] 
whatprocessdesign[1-0] 
whatprocessfielding[1  -0] 
whatprocessSWEngSuppt[1  -0] 
whatproductscustom[1  -0] 
whatproductscommoncust[1  -0] 


Estimate 

4.0654463 

0.5220941 

1.4998382 

-0.350248 

0.808367 

-0.607973 

-0.454671 

0.7227588 

-0.592745 

0.4961645 

-0.372512 

0.4776907 


Std  Error 
0.189091 
0.302864 
0.457642 
0.299505 
0.310334 
0.480358 
0.195726 
0.186669 
0.223661 
0.187698 
0.17877 
0.192216 


t  Ratio  Prob>|t| 

21.50  <.0001 

1.72  0.0894 

3.28  0.0017 

-1.17  0.2464 

2.60  0.0113 

-1.27  0.2101 

-2.32  0.0233 

3.87  0.0003 

-2.65  0.0101 

2.64  0.0102 

-2.08  0.0411 

2.49  0.0155 


Effect  Tests 

Source 

whattypesystemsavionics 

whattypeshrinkutilities 

whattypecomponentOS 

whattypeenterpriseacctng 

whattypeenterpriseOES 

wh  atprocessreq  u  i  rements 

whatprocessdesign 

whatprocessfielding 

whatprocessSWEngSuppt 

whatproductscustom 

whatproductscommoncust 


Nparm  DF 

1  1 

1  1 

1  1 

1  1 

1  1 

1  1 

1  1 

1  1 

1  1 

1  1 

1  1 


Sum  of  Squares 
1.1549131 
4.1742952 
0.5314815 
2.6369638 
0.6225647 
2.0972151 
5.8262655 
2.7296085 
2.7156902 
1.6874726 
2.4002796 


F  Ratio  Prob  >  F 

2.9717  0.0894 

10.7408  0.0017 

1.3675  0.2464 

6.7851  0.0113 

1.6019  0.2101 

5.3963  0.0233 

14.9914  0.0003 

7.0235  0.0101 

6.9877  0.0102 

4.3420  0.041 1 

6.1761  0.0155 


LangCult  =  4.07  +  sys-avia(0.52)  +  shrink-util(1.50)  +  comp-OS(-0.35)  +  ent-acct(0.81)  +  ent-OES(-0.61)  +  proc-req(-0.45) 
+  proc-des(0.72)  +  proc-field(-0.59)  +  proc-SWEngSup(0.50)  +  prod-cust(-0.37)  +  prod-comcust(0.48) 
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Stepwise  Fit  -  Old  Survey  Data  -  Consequences  (TurfWar) 

Response: 

Column  59 


Stepwise  Regression  Control 

Prob  to  Enter  0.250 

Prob  to  Leave  0.250 

Direction: 

Rules: 


Current  Estimates 

SSE  DFE 

46.900052  63 


MSE  RSquare 

0.7444453  0.5670 


RSquare  Adj  Cp  AIC 

0.4776  0.5035653  -10.1756 


Lock 

X 


X  Intercept 

X  whattypesystemsavionics{0-1} 

whattypesystemsembedded{0-1 } 
whattypesystemscommunications{0-1} 
X  whattypesystemsdevice{0-1} 

whattypeshrinkbusiness{0-1} 
whattypeshrinkutilities{0-1 } 

X  whattypeshrinkinternet{0-1} 

whattypecomponentdomain{0-1} 

whattypecomponentCASE{0-1} 

X  whattypecomponentclass{0-1} 

X  whattypecomponentOS{0-1} 

whattypecomponentdevelopment{0-1} 
X  whattypeenterpriseacctng{0-1} 

X  whattypeenterprisemanufact{0-1} 

whattypeenterprisepayroll{0-1 } 
whattypeenterpriseOES{0- 1 } 
whattypeenterprisescripting{0-1} 
whattypeenterpriseweb{0-1 } 
whatprocessrequirements{0-1} 
whatprocessdesign{0- 1 } 
whatprocesstesting{0-1} 
whatprocessmaintenance{0-1} 

X  whatprocessreengineering{0-1} 

X  whatprocessappsuppt{0-1} 

whatprocesstraining{0-1} 

X  whatprocessspecification{0-1} 

whatprocessdocumentation{0-1 } 
whatprocesscoding{0- 1 } 
whatprocessfielding{0-1} 
whatprocessCM{0-1} 

X  whatprocesstoolsuppt{0-1} 

X  whatprocessSWEngSuppt{0-1} 

whatproductscustom{0-1 } 

X  whatproductsCOTS{0-1} 

whatproductscommoncust{0-1} 
whatproductsnone{0-1 } 


Step  History 

Step  Parameter 

1  whatprocessSWEngSuppt{0-1} 

2  whatprocessappsuppt{0-1} 

3  whatprocesstoolsuppt{0-1} 

4  whatproductsCOTS{0-1} 

5  whattypeenterprisemanufact{0-1} 

6  whattypesystemsdevice{0-1} 

7  whatprocessreengineering{0-1} 

8  whattypeshrinkinternet{0-1} 

9  whattypeenterpriseacctng{0-1} 

10  whattypecomponentOS{0-1} 

11  whatprocessspecification{0-1} 

12  whattypesystemsavionics{0-1} 

13  whattypecomponentclass{0-1} 


Action 

Entered 

Entered 

Entered 

Entered 

Entered 

Entered 

Entered 

Entered 

Entered 

Entered 

Entered 

Entered 

Entered 


Estimate  nDF 

5.53402614  1 

0.31559089  1 

0  1 

0  1 

-0.6437914  1 

0  1 

0  1 

-0.6609301  1 

0  1 

0  1 

0.41165857  1 

0.38193024  1 

0  1 

-0.4355382  1 

-0.7723433  1 

0  0 

0  1 

0  1 

0  1 

0  1 

0  1 

0  1 

0  1 

0.44378108  1 

0.99328021  1 

0  1 

0.26504064  1 

0  1 

0  1 

0  1 

0  1 

-0.6037973  1 

-0.4751488  1 

0  1 

-0.469777  1 

0  1 

0  1 


SS 

0 

1.637139 

0.797578 

0.384485 

5.720934 

0.165339 

0.801746 

4.686187 

0.092164 

0.240955 

1.085066 

2.359945 

0.023116 

2.889767 

1.841513 

0 

0.108468 

0.382585 

0.87859 

0.471544 

0.862389 

0.001193 

0.201201 

4.671428 

14.60499 

0.547117 

2.308251 

0.077768 

0.277806 

0.271972 

0.054212 

10.92175 

8.936858 

0.407721 

8.933564 

0.007618 

0.09953 


"Sig  Prob" 
0.0134 
0.0000 
0.0145 
0.0204 
0.0322 
0.0478 
0.0873 
0.1104 
0.0968 
0.1425 
0.1361 
0.1563 
0.2318 


Seq  SS 
8.543966 
20.66413 
6.260439 
5.274857 
4.254521 
3.470346 
2.500749 
2.124813 
2.242833 
1.711251 
1.735637 
1.543031 
1.085066 


RSquare 

0.0789 

0.2697 

0.3275 

0.3762 

0.4154 

0.4475 

0.4706 

0.4902 

0.5109 

0.5267 

0.5427 

0.5570 

0.5670 


"F  Ratio"  "Prob>F" 

0.000  1.0000 

2.199  0.1431 

1.073  0.3044 

0.512  0.4768 

7.685  0.0073 

0.219  0.6412 

1.078  0.3031 

6.295  0.0147 

0.122  0.7280 

0.320  0.5735 

1.458  0.2318 

3.170  0.0798 

0.031  0.8618 

3.882  0.0532 

2.474  0.1208 

0.144  0.7059 

0.510  0.4779 

1.184  0.2808 

0.630  0.4305 

1.161  0.2854 

0.002  0.9685 

0.267  0.6071 

6.275  0.0148 

19.619  0.0000 

0.732  0.3956 

3.101  0.0831 

0.103  0.7494 

0.369  0.5455 

0.362  0.5498 

0.072  0.7897 

14.671  0.0003 

12.005  0.0010 

0.544  0.4637 

12.000  0.0010 

0.010  0.9204 

0.132  0.7178 


Cp  p 

32.306  2 

12.495  3 

7.8868  4 

4.3191  5 

1 .8284  6 

0.1655  7 

-0.474  8 

-0.717  9 

-1.084  10 

-0.89  1 1 

-0.722  12 

-0.351  13 

0.5036  14 


Appendix  C 


Response  Column  59 
Actual  by  Predicted  Plot 


Column  59  Predicted  P<.0001  RSq=0.57 
RMSE=0.8628 


Summary  of  Fit 


RSquare 
RSquare  Adj 
Root  Mean  Square  Error 
Mean  of  Response 
Observations  (or  Sum  Wgts) 
Analysis  of  Variance 
Source  DF 

Model  13 

Error  63 

C.  Total  76 

Lack  Of  Fit 

Source  DF 

Lack  Of  Fit  23 

Pure  Error  40 

Total  Error  63 


0.56699 

0.477639 

0.862812 

4.61039 


Sum  of  Squares 
61.41164 
46.90005 
108.31169 


Sum  of  Squares 
19.284667 
27.615385 
46.900052 


Parameter  Estimates 

Term 

Intercept 

whattypesystemsavionics[1  -0] 
whattypesystemsdevice[1  -0] 
whattypeshrinkinternet[1  -0] 
whattypecomponentclass[1  -0] 
whattypecomponentOS[1  -0] 
whattypeenterpriseacctng[1  -0] 
whattypeenterprisemanufact[1  -0] 
whatprocessreengineering[1  -0] 
whatprocessappsuppt[1  -0] 
whatprocessspecification[1  -0] 
whatprocesstoolsuppt[1  -0] 
whatprocessSWEngSuppt[1  -0] 


whatproductsCOTS[1-0] 

Effect  Tests 

Source  Nparm 

whattypesystemsavionics  1 

whattypesystemsdevice  1 

whattypeshrinkinternet  1 

whattypecomponentclass  1 

whattypecomponentOS  1 

whattypeenterpriseacctng  1 

whattypeenterprisemanufact  1 

whatprocessreengineering  1 

whatprocessappsuppt  1 

whatprocessspecification  1 

whatprocesstoolsuppt  1 

whatprocessSWEngSuppt  1 

whatproductsCOTS  1 


Mean  Square  F  Ratio 

4.72397  6.3456 

0.74445  Prob  >  F 

<.0001 


Mean  Square  F  Ratio 

0.838464  1.2145 

0.690385  Prob  >  F 

0.2883 
Max  RSq 
0.7450 


Estimate 
4.2839816 
-0.631182 
1.2875828 
1.3218603 
-0.823317 
-0.76386 
0.8710764 
1.5446865 
-0.887562 
-1 .98656 
-0.530081 
1.2075947 
0.9502976 
0.939554 


Std  Error 
0.152293 
0.425626 
0.46447 
0.526856 
0.681955 
0.429022 
0.442121 
0.982131 
0.354316 
0.448505 
0.301035 
0.315276 
0.274273 
0.271223 


t  Ratio 
28.13 
-1.48 
2.77 
2.51 
-1.21 
-1.78 
1.97 
1.57 
-2.51 
-4.43 
-1.76 
3.83 
3.46 
3.46 


Prob>|t| 

<.0001 

0.1431 

0.0073 

0.0147 

0.2318 

0.0798 

0.0532 

0.1208 

0.0148 

<.0001 

0.0831 

0.0003 

0.0010 

0.0010 


DF  Sum  of  Squares 

1  1.637139 

1  5.720934 

1  4.686187 

1  1 .085066 

1  2.359945 

1  2.889767 

1  1.841513 

1  4.671428 

1  14.604993 

1  2.308251 

1  10.921755 

1  8.936858 

1  8.933564 


F  Ratio  Prob  >  F 

2.1991  0.1431 

7.6848  0.0073 

6.2949  0.0147 

1.4575  0.2318 

3.1701  0.0798 

3.8818  0.0532 

2.4737  0.1208 

6.2750  0.0148 

19.6186  <.0001 

3.1006  0.0831 

14.6710  0.0003 

12.0047  0.0010 

12.0003  0.0010 


TurfWar  =  4.28  +  sys-avia(-0.63)  +  sys-dev(1.29)  +  shrink-int(1 .32)  +  comp-class(-0.82)  +  comp-OS(-0.76)  +  ent- 
acct(0.87)  +  ent-mnft(1.54)  +  proc-reeng(-0.89)  +  proc-appsup(-1.99)  +  proc-spec(-0.53)  +  proc-toolsup(1.21)  +  proc- 
SWEngSup(0.95)  +  prod-COTS(0.94) 
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Stepwise  Fit  -  Old  Survey  Data  -  Consequences  (FailLikely) 

Response: 

Column  59 


Stepwise  Regression  Control 

Prob  to  Enter  0.250 

Prob  to  Leave  0.250 

Direction: 

Rules: 


Current  Estimates 

SSE  DFE 

95.835942  70 


MSE  RSquare 

1.3690849  0.3647 


RSquare  Adj 
0.2921 


Cp  AIC 

-6.494822  33.26201 


Lock 

X 


X  Intercept 

whattypesystemsavionics{0-1} 
whattypesystemsembedded{0-1 } 
whattypesystemscommunications{0-1} 
X  whattypesystemsdevice{0-1} 

whattypeshrinkbusiness{0-1} 
whattypeshrinkutilities{0-1 } 

X  whattypeshrinkinternet{0-1} 

whattypecomponentdomain{0-1} 
whattypecomponentCASE{0-1} 
whattypecomponentclass{0- 1 } 
whattypecomponentOS{0-1} 
whattypecomponentdevelopment{0-1} 
whattypeenterpriseacctng{0-1 } 
whattypeenterprisemanufact{0-1 } 
whattypeenterprisepayroll{0-1 } 
whattypeenterpriseOES{0- 1 } 
whattypeenterprisescripting{0-1} 
whattypeenterpriseweb{0-1 } 

X  whatprocessrequirements{0-1} 
whatprocessdesign{0- 1 } 
whatprocesstesting{0- 1 } 
whatprocessmaintenance{0-1} 

X  whatprocessreengineering{0-1} 

whatprocessappsuppt{0-1 } 
whatprocesstraining{0-1} 
whatprocessspecification{0-1 } 
whatprocessdocumentation{0-1 } 
whatprocesscoding{0- 1 } 
whatprocessfielding{0-1} 

X  whatprocessCM{0-1} 

X  whatprocesstoolsuppt{0-1} 

X  whatprocessSWEngSuppt{0-1} 

whatproductscustom{0-1 } 

X  whatproductsCOTS{0-1} 

whatproductscommoncust{0-1} 
whatproductsnone{0-1 } 


Step  History 

Step  Parameter 

1  whatprocesstoolsuppt{0-1} 

2  whatprocessCM{0-1} 

3  whatprocessSWEngSuppt{0-1} 

4  whatproductsCOTS{0-1} 

5  whatprocesstraining{0-1} 

6  whattypeshrinkinternet{0-1} 

7  whatprocessrequirements{0-1} 

8  whatprocessfielding{0-1} 

9  whatprocessreengineering{0-1} 

10  whatprocesstraining{0-1} 

1 1  whattypesystemsdevice{0- 1 } 

12  whatprocessfielding{0-1} 


Action 

Entered 

Entered 

Entered 

Entered 

Entered 

Entered 

Entered 

Entered 

Entered 

Removed 

Entered 

Removed 


Estimate  nDF 

5.40906375  1 

0  1 

0  1 

0  1 

-0.5178269  1 

0  1 

0  1 

-0.7893818  1 

0  1 

0  1 

0  1 

0  1 

0  1 

0  1 

0  1 

0  1 

0  1 

0  1 

0  1 

0.34959664  1 

0  1 

0  1 

0  1 

0.44994513  1 

0  1 

0  1 

0  1 

0  1 

0  1 

0  1 

0.54582382  1 

-0.711711  1 

-0.4748514  1 

0  1 

-0.4108839  1 

0  1 

0  1 


"Sig  Prob" 
0.0085 
0.0035 
0.0490 
0.1072 
0.0660 
0.1502 
0.1247 
0.1149 
0.1793 
0.2713 
0.1591 
0.2518 


Seq  SS 
13.06474 
14.67609 
6.242145 
4.053617 
5.137712 
3.073144 
3.440048 
3.554579 
2.536763 
1.694851 
2.760612 
1.819789 


SS 

0 

0.000212 

0.576873 

0.313122 

3.722764 

0.573727 

0.215409 

8.579286 

0.3406 

0.624582 

0.705971 

0.086229 

0.782647 

0.976331 

0.865661 

0.705971 

1.040914 

0.219048 

0.420508 

4.172481 

0.14806 

0.630976 

0.638896 

6.723505 

0.005849 

0.764498 

0.001768 

1.125733 

0.66914 

1.819789 

10.29045 

15.76179 

8.622342 

0.677889 

7.571181 

0.874191 

0.203204 


RSquare 

0.0866 

0.1839 

0.2253 

0.2521 

0.2862 

0.3066 

0.3294 

0.3529 

0.3697 

0.3585 

0.3768 

0.3647 


"F  Ratio"  "Prob>F" 

0.000  1.0000 

0.000  0.9902 

0.418  0.5202 

0.226  0.6359 

2.719  0.1036 

0.416  0.5213 

0.155  0.6946 

6.266  0.0146 

0.246  0.6214 

0.453  0.5033 

0.512  0.4767 

0.062  0.8039 

0.568  0.4536 

0.710  0.4023 

0.629  0.4305 

0.512  0.4767 

0.758  0.3871 

0.158  0.6922 

0.304  0.5831 

3.048  0.0852 

0.107  0.7448 

0.457  0.5011 

0.463  0.4985 

4.911  0.0299 

0.004  0.9484 

0.555  0.4589 

0.001  0.9716 

0.820  0.3683 

0.485  0.4884 

1.336  0.2518 

7.516  0.0078 

11.513  0.0011 

6.298  0.0144 

0.492  0.4856 

5.530  0.0215 

0.635  0.4282 

0.147  0.7030 


Cp  p 

3.3693  2 

-2.977  3 

-4.528  4 

-4.833  5 

-5.755  6 

-5.503  7 

-5.459  8 

-5.481  9 

-4.924  10 

-5.96  9 

-5.53  10 

-6.495  9 
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Response  Column  59 
Actual  by  Predicted  Plot 


1 

3 


1  2  3  4  5  6  7  8 

Column  59  Predicted  P<.0001  RSq=0.36 
RMSE=1.1701 


Summary  of  Fit 


RSquare  0.364739 

RSquare  Adj  0.292138 

Root  Mean  Square  Error  1.170079 

Mean  of  Response  4.164557 

Observations  (or  Sum  Wgts)  79 


Analysis  of  Variance 


Source  DF 

Model  8 

Error  70 

C.  Total  78 


Sum  of  Squares 
55.02482 
95.83594 
150.86076 


Mean  Square  F  Ratio 

6.87810  5.0239 

1.36908  Prob  >  F 

<.0001 


Lack  Of  Fit 


Source  DF 

Lack  Of  Fit  17 

Pure  Error  53 

Total  Error  70 


Sum  of  Squares 
27.390540 
68.445402 
95.835942 


Mean  Square  F  Ratio 

1.61121  1.2476 

1.29142  Prob  >  F 

0.2628 
Max  RSq 
0.5463 


Parameter  Estimates 


Term 

Estimate 

Std  Error 

t  Ratio  Prob>|t| 

Intercept 

3.8497742 

0.183944 

20.93  <.0001 

whattypesystemsdevice[1  -0] 

1.0356539 

0.628054 

1.65  0.1036 

whattypeshrinkinternet[1  -0] 

1.5787637 

0.630677 

2.50  0.0146 

whatprocessrequirementsjl  -0] 

-0.699193 

0.400512 

-1 .75  0.0852 

whatprocessreengineering[1-0] 

-0.89989 

0.406076 

-2.22  0.0299 

whatprocessCM[1-0] 

-1.091648 

0.398181 

-2.74  0.0078 

whatprocesstoolsuppt[1  -0] 

1.4234221 

0.419514 

3.39  0.0011 

whatprocessSWEngSuppt[1  -0] 

0.9497029 

0.378434 

2.51  0.0144 

whatproductsCOTSjl  -0] 

0.8217679 

0.349448 

2.35  0.0215 

Effect  Tests 

Source 

Nparm  DF 

Sum  of  Squares 

F  Ratio 

Prob  >  F 

whattypesystemsdevice 

1  1 

3.722764 

2.7192 

0.1036 

whattypeshrinkinternet 

1  1 

8.579286 

6.2664 

0.0146 

whatprocessrequirements 

1  1 

4.172481 

3.0476 

0.0852 

whatprocessreengineering 

1  1 

6.723505 

4.9109 

0.0299 

whatprocessCM 

1  1 

10.290453 

7.5163 

0.0078 

whatprocesstoolsuppt 

1  1 

15.761794 

11.5126 

0.0011 

whatprocessSWEngSuppt 

1  1 

8.622342 

6.2979 

0.0144 

whatproductsCOTS 

1  1 

7.571181 

5.5301 

0.0215 

FailLikely  =  3.85  +  sys-dev(1.04)  +  shrink-int(1 .58)  +  proc-req(-0.70)  +  proc-reeng(-0.90)  +  proc-CM(-1 .09)  +  proc- 
toolsup(1.42)  +  proc-SWEngSup(0.95)  +  prod-COTS(0.82) 
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Stepwise  Fit  -  Old  Survey  Data  -  Consequences  (RespCust) 

Response: 

Column  59 


Stepwise  Regression  Control 

Prob  to  Enter  0.250 

Prob  to  Leave  0.250 

Direction: 

Rules: 


Current  Estimates 

SSE  DFE 

112.51421  73 


MSE  RSquare 

1.5412905  0.2159 


RSquare  Adj 
0.1514 


Cp  AIC 

-13.2317  41.28423 


Lock 

X 


X  Intercept 

whattypesystemsavionics{0-1} 
whattypesystemsembedded{0-1 } 
whattypesystemscommunications{0-1} 
whattypesystemsdevice{0-1} 
whattypeshrinkbusiness{0-1} 
whattypeshrinkutilities{0-1 } 
whattypeshrinkinternet{0-1 } 

X  whattypecomponentdomain{0-1} 

whattypecomponentCASE{0-1} 
whattypecomponentclass{0- 1 } 
whattypecomponentOS{0-1} 
whattypecomponentdevelopment{0-1} 
whattypeenterpriseacctng{0-1 } 
whattypeenterprisemanufact{0-1 } 
whattypeenterprisepayroll{0-1 } 

X  whattypeenterpriseOES{0-1} 

whattypeenterprisescripting{0-1} 
whattypeenterpriseweb{0-1 } 
whatprocessrequirements{0-1} 
whatprocessdesign{0- 1 } 
whatprocesstesting{0-1} 

X  whatprocessmaintenance{0-1} 

whatprocessreengineering{0-1} 

X  whatprocessappsuppt{0-1} 

whatprocesstraining{0-1} 
whatprocessspecification{0-1 } 
whatprocessdocumentation{0-1 } 
whatprocesscoding{0- 1 } 
whatprocessfielding{0-1} 
whatprocessCM{0-1} 
whatprocesstoolsuppt{0-1} 
whatprocessSWEngSuppt{0- 1 } 

X  whatproductscustom{0-1} 

whatproductsCOTS{0-1} 

X  whatproductscommoncust{0-1} 

whatproductsnone{0-1 } 


Step  History 

Step  Parameter 

1  whatprocessSWEngSuppt{0-1} 

2  whattypeenterpriseOES{0-1} 

3  whatproductscommoncust{0-1} 

4  whatprocessappsuppt{0-1} 

5  whattypecomponentdomain{0-1} 

6  whatproductscustom{0-1} 

7  whatprocessSWEngSuppt{0-1} 

8  whatprocessmaintenance{0-1} 


Action 

Entered 

Entered 

Entered 

Entered 

Entered 

Entered 

Removed 

Entered 


Estimate  nDF 

4.14377034  1 

0  1 

0  1 

0  1 

0  1 

0  1 

0  1 

0  1 

0.43479846  1 

0  1 

0  1 

0  1 

0  1 

0  1 

0  1 

0  1 

-0.9214312  1 

0  1 

0  1 

0  1 

0  1 

0  1 

-0.1980136  1 

0  1 

0.60987965  1 

0  1 

0  1 

0  1 

0  1 

0  1 

0  1 

0  1 

0  1 

0.26185915  1 

0  1 

0.39251674  1 

0  1 


SS 

0 

0.030685 
0.03554 
0.853077 
0.070901 
0.038755 
1.018881 
1.04921 
3.385631 
0.714462 
0.000259 
0.048652 
0.198655 
1 .376875 
0.01516 
0.000259 
6.368249 
1 .983895 
0.002705 
0.686152 
0.001957 
1.400097 
2.151545 
0.438128 
10.60416 
1 .05205 
0.013479 
0.130216 
0.56503 
0.07733 
0.719331 
0.021224 
1.38844 
3.960955 
0.115988 
7.232801 
0.175587 


"Sig  Prob" 
0.0090 
0.0957 
0.0664 
0.1908 
0.1689 
0.2292 
0.3226 
0.2412 


Seq  SS 
12.10431 
4.680478 
5.531956 
2.751942 
3.01066 
2.27908 
1.536676 
2.151545 


RSquare 

0.0844 

0.1170 

0.1555 

0.1747 

0.1957 

0.2116 

0.2009 

0.2159 


"F  Ratio" 
0.000 
0.020 
0.023 
0.550 
0.045 
0.025 
0.658 
0.678 
2.197 
0.460 
0.000 
0.031 
0.127 
0.892 
0.010 
0.000 
4.132 
1.292 
0.002 
0.442 
0.001 
0.907 
1.396 
0.281 
6.880 
0.680 
0.009 
0.083 
0.363 
0.050 
0.463 
0.014 
0.900 
2.570 
0.074 
4.693 
0.113 


Cp 

-14.38 

-14.58 

-15.17 

-14.46 

-13.87 

-12.94 

-14.22 

-13.23 


'Prob>F" 

1.0000 

0.8889 

0.8805 

0.4607 

0.8319 

0.8753 

0.4200 

0.4131 

0.1426 

0.4997 

0.9898 

0.8604 

0.7222 

0.3481 

0.9218 

0.9898 

0.0457 

0.2594 

0.9669 

0.5084 

0.9719 

0.3440 

0.2412 

0.5974 

0.0106 

0.4125 

0.9263 

0.7735 

0.5485 

0.8245 

0.4983 

0.9075 

0.3461 

0.1132 

0.7860 

0.0336 

0.7382 


P 

2 

3 

4 

5 

6 
7 
6 
7 


Appendix  C 


Response  Column  59 
Actual  by  Predicted  Plot 


<5- 

05 

in  4—- 

Is- 

O 

O  2- 


1  2  3  4  5  6  7 

Column  59  Predicted  P=0.0057  RSq=0.22 
RMSE=1.2415 


Summary  of  Fit 


RSquare  0.215861 

RSquare  Adj  0.151411 

Root  Mean  Square  Error  1 .241487 

Mean  of  Response  4.1 375 

Observations  (or  Sum  Wgts)  80 


Analysis  of  Variance 


Source  DF 

Model  6 

Error  73 

C.  Total  79 


Sum  of  Squares 
30.97329 
112.51421 
143.48750 


Mean  Square  F  Ratio 

5.16222  3.3493 

1.54129  Prob  >  F 

0.0057 


Lack  Of  Fit 

Source  DF  Sum  of  Squares 

Lack  Of  Fit  10  17.23224 

Pure  Error  63  95.28196 

Total  Error  73  112.51421 


Mean  Square  F  Ratio 

1.72322  1.1394 

1.51241  Prob  >  F 

0.3483 
Max  RSq 
0.3360 


Parameter  Estimates 

Term 

Intercept 

whattypecomponentdomain[1  -0] 
whattypeenterpriseOES[1  -0] 
whatprocessmaintenance[1  -0] 
whatprocessappsuppt[1  -0] 
Whatproductscustom[1  -0] 
whatproductscommoncust[1  -0] 


Estimate 

4.7233795 

-0.869597 

1.8428624 

0.3960273 

-1.219759 

-0.523718 

-0.785033 


Std  Error 
0.31459 
0.586733 
0.90662 
0.335191 
0.465027 
0.326693 
0.362391 


t  Ratio 
15.01 
-1.48 
2.03 
1.18 
-2.62 
-1.60 
-2.17 


Prob>|t| 

<.0001 

0.1426 

0.0457 

0.2412 

0.0106 

0.1132 

0.0336 


Effect  Tests 

Source 

whattypecomponentdomain 

whattypeenterpriseOES 

whatprocessmaintenance 

whatprocessappsuppt 

Whatproductscustom 

whatproductscommoncust 


Nparm  DF 

1  1 

1  1 

1  1 


Sum  of  Squares 
3.385631 
6.368249 
2.151545 
10.604163 
3.960955 
7.232801 


F  Ratio  Prob  >  F 

2.1966  0.1426 

4.1318  0.0457 

1.3959  0.2412 

6.8801  0.0106 

2.5699  0.1132 

4.6927  0.0336 


ResponseCustomer  =  4.72  +  comp-domain(-0.87)  +  ent-OES(1.84)  +  proc-maint(0.40)  +  proc-appsup(-1.22)  +  prod-cust(- 
0.52)  +  prod-comcust(-0.79) 
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Stepwise  Fit  -  Old  Survey  Data  -  Consequences  (ResponseOrg) 

Response: 

Column  59 


Stepwise  Regression  Control 

Prob  to  Enter  0.250 

Prob  to  Leave  0.250 

Direction: 

Rules: 

Current  Estimates 


SSE 

DFE  MSE  RSquare 

RSquare  Adj  Cp 

AIC 

150.46514 

75  2.0062019 

0.2090 

0.1563  -11.77099 

62.16187 

Lock 

Entered 

Parameter 

Estimate  nDF 

SS 

"F  Ratio" 

"Prob>F" 

X 

X 

Intercept 

4.25908726  1 

0 

0.000 

1.0000 

whattypesystemsavionics{0-1 } 

0  1 

0.864092 

0.427 

0.5153 

whattypesystemsembedded{0-1 } 

0  1 

0.204138 

0.101 

0.7521 

whattypesystemscommunications{0-1} 

0  1 

0.796818 

0.394 

0.5322 

whattypesystemsdevice{0- 1 } 

0  1 

1.042423 

0.516 

0.4747 

whattypeshrinkbusiness{0-1} 

0  1 

0.263623 

0.130 

0.7196 

whattypeshrinkutilities{0-1 } 

0  1 

0.205052 

0.101 

0.7515 

whattypeshrinkinternet{0- 1 } 

0  1 

1.56774 

0.779 

0.3803 

whattypecomponentdomain{0-1} 

0  1 

1 .32839 

0.659 

0.4195 

whattypecomponentCASE{0-1 } 

0  1 

2.153906 

1.075 

0.3033 

whattypecomponentclass{0- 1 } 

0  1 

1 .54323 

0.767 

0.3840 

whattypecomponentOS{0-1 } 

0  1 

0.066461 

0.033 

0.8570 

whattypecomponentdevelopment{0-1} 

0  1 

2.159639 

1.078 

0.3026 

whattypeenterpriseacctng{0-1} 

0  1 

1.844384 

0.918 

0.3410 

whattypeenterprisemanufact{0-1} 

0  1 

1.153936 

0.572 

0.4519 

whattypeenterprisepayroll{0-1} 

0  1 

1 .54323 

0.767 

0.3840 

X 

whattypeenterpriseOES{0-1} 

-0.912338  1 

6.288016 

3.134 

0.0807 

whattypeenterprisescripting{0-1} 

0  1 

2.05496 

1.025 

0.3147 

whattypeenterpriseweb{0-1 } 

0  1 

0.425712 

0.210 

0.6481 

whatprocessrequirements{0-1} 

0  1 

0.033119 

0.016 

0.8988 

whatprocessdesign{0-1} 

0  1 

0.114257 

0.056 

0.8132 

whatprocesstesting{0-1} 

0  1 

1.733468 

0.862 

0.3561 

X 

whatprocessmaintenance{0-1} 

-0.3700264  1 

7.874845 

3.925 

0.0512 

whatprocessreengineering{0-1 } 

0  1 

0.409686 

0.202 

0.6544 

X 

whatprocessappsuppt{0-1 } 

0.63692046  1 

9.852647 

4.911 

0.0297 

whatprocesstraining{0-1} 

0  1 

0.0035 

0.002 

0.9670 

whatprocessspecification{0-1 } 

0  1 

0.042193 

0.021 

0.8858 

whatprocessdocumentation{0-1 } 

0  1 

0.406443 

0.200 

0.6557 

X 

whatprocesscoding{0-1} 

-0.3285747  1 

4.040448 

2.014 

0.1600 

whatprocessfielding{0-1} 

0  1 

1.354446 

0.672 

0.4149 

whatprocessCM{0-1} 

0  1 

1.452662 

0.721 

0.3984 

whatprocesstoolsuppt{0-1} 

0  1 

0.88543 

0.438 

0.5101 

X 

whatprocessS  WEngSuppt{0- 1 } 

0.3835856  1 

8.178324 

4.077 

0.0471 

whatproductscustom{0-1 } 

0  1 

0.604958 

0.299 

0.5863 

whatproductsCOTS{0-1} 

0  1 

0.851032 

0.421 

0.5185 

whatproductscommoncust{0-1} 

0  1 

0.80964 

0.400 

0.5289 

whatproductsnone{0-1} 

0  1 

0.000017 

0.000 

0.9977 

Step  History 

Step 

Parameter 

Action 

"Sig  Prob”  Seq  SS 

RSquare 

Cp 

P 

1 

whatprocessSWEngSuppt{0-1 } 

Entered 

0.0051  18.05159 

0.0949 

-11.52 

2 

2 

whatprocesscoding{0-1 } 

Entered 

0.1157  5.411658 

0.1233 

-11.57 

3 

3 

whattypeenterpriseOES{0-1} 

Entered 

0.1599  4.250523 

0.1457 

-11.19 

4 

4 

whatprocessappsuppt{0-1 } 

Entered 

0.1613  4.168463 

0.1676 

-10.78 

5 

5 

whatprocessmaintenance{0-1} 

Entered 

0.0512  7.874845 

0.2090 

-11.77 

6 

Appendix  C 


Response  Column  59 
Actual  by  Predicted  Plot 


Column  59  Predicted  P=0.0030  RSq=0.21 
RMSE=1.4164 


Summary  of  Fit 


RSquare  0.209003 

RSquareAdj  0.15627 

Root  Mean  Square  Error  1.416405 

Mean  of  Response  4.148148 

Observations  (or  Sum  Wgts)  81 


Analysis  of  Variance 

Source  DF 

Model  5 

Error  75 

C.  Total  80 

Lack  Of  Fit 

Source 
Lack  Of  Fit 
Pure  Error 
Total  Error 

Max  RSq 
0.2968 


DF  Sum  of  Squares 

6  16.70106 

69  133.76408 

75  150.46514 


Mean  Square  F  Ratio 

2.78351  1.4358 

1.93861  Prob  >  F 

0.2136 


Sum  of  Squares  Mean  Square  F  Ratio 

39.75708  7.95142  3.9634 

150.46514  2.00620  Prob  >  F 

190.22222  0.0030 


Parameter  Estimates 


Term 

Estimate 

Std  Error 

t  Ratio  Prob>|t| 

Intercept 

3.6686542 

0.444772 

8.25  <.0001 

whattypeenterpriseOES[1  -0] 

1.8246761 

1.030662 

1.77  0.0807 

whatprocessmaintenance[1  -0] 

0.7400527 

0.373533 

1.98  0.0512 

whatprocessappsuppt[1  -0] 

-1 .273841 

0.574812 

-2.22  0.0297 

whatprocesscoding[1-0] 

0.6571494 

0.463059 

1.42  0.1600 

whatprocessSWEngSuppt[1  -0] 

-0.767171 

0.379968 

-2.02  0.0471 

Effect  Tests 

Source 

Nparm  DF  Sum  of  Squares 

F  Ratio 

Prob  >  F 

whattypeenterpriseOES 

1  1 

6.2880156 

3.1343 

0.0807 

whatprocessmaintenance 

1  1 

7.8748445 

3.9253 

0.0512 

whatprocessappsuppt 

1  1 

9.8526472 

4.9111 

0.0297 

whatprocesscoding 

1  1 

4.0404484 

2.0140 

0.1600 

whatprocessSWEngSuppt 

1  1 

8.1783235 

4.0765 

0.0471 

ResponseOrg  =  3.67  +  ent-OES(1.82)  +  proc-maint(0.74) 
0.77) 

+  proc-appsup(-1.27)  +  proc-coding(0.66)  +  proc-SWEngSup(- 
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Appendix  D  -  Combined  Survey  Data  Consequence  Models 

Stepwise  Fit  -  Combined  Survey  Data  -  Consequences  (Cost) 

Response: 

Column  59 

Stepwise  Regression  Control 

Prob  to  Enter  0.250 

Prob  to  Leave  0.250 

Direction: 

Rules: 


Current  Estimates 

SSE  DFE 

247.07064  112 


MSE  RSquare 

2.2059879  0.2862 


RSquare  Adj  Cp  AIC 

0.1970  0.7879503  114.5169 


Lock 

X 


X  Intercept 

whattypesystemsavionics{0-1 } 
whattypesystemsembedded{0-1} 
whattypesystemscommunications{0-1} 
whattypesystemsdevice{0-1} 

X  whattypeshrinkbusiness{0-1} 

whattypeshrinkutilities{0-1 } 

X  whattypeshrinkinternet{0-1} 

whattypecomponentdomain{0-1} 

X  whattypecomponentCASE{0-1} 

whattypecomponentclass{0- 1 } 
whattypecomponentOS{0-1} 
whattypecomponentdevelopment{0-1} 
whattypeenterpriseacctng{0-1 } 

X  whattypeenterprisemanufact{0-1} 

X  whattypeenterprisepayroll{0-1} 

whattypeenterpriseOES{0- 1 } 
whattypeenterprisescripting{0-1} 

X  whattypeenterpriseweb{0-1} 

whatprocessrequirements{0-1} 

X  whatprocessdesign{0-1} 

X  whatprocesstesting{0-1} 

X  whatprocessmaintenance{0-1} 

whatprocessreengineering{0-1 } 
whatprocessappsuppt{0-1 } 
whatprocesstraining{0-1} 

X  whatprocessspecification{0-1} 

whatprocessdocumentation{0-1 } 
whatprocesscoding{0- 1 } 
whatprocessfielding{0-1} 

X  whatprocessCM{0-1} 

whatprocesstoolsuppt{0-1} 

X  whatprocessSWEngSuppt{0-1} 

whatprocessnone{0-1 } 
whatproductscustom{0-1 } 

X  whatproductsCOTS{0-1} 

X  whatproductscommoncust{0-1} 

whatproductsnone{0-1 } 


Step  History 

Step  Parameter 

1  whattypecomponentCASE{0-1} 

2  whattypeenterprisepayroll{0-1} 

3  whattypeenterpriseweb{0-1} 

4  whattypeshrinkbusiness{0-1} 

5  whattypeshrinkinternet{0-1} 

6  whatprocessspecification{0-1} 

7  whatprocessCM{0-1} 

8  whattypeenterprisemanufact{0-1} 

9  whatprocesstesting{0-1} 

10  whatprocessSWEngSuppt{0-1} 

11  whatprocessmaintenance{0-1} 

12  whatproductsCOTS{0-1} 

13  whatprocessdesign{0-1} 

14  whatproductscommoncust{0-1} 


Action 

Entered 

Entered 

Entered 

Entered 

Entered 

Entered 

Entered 

Entered 

Entered 

Entered 

Entered 

Entered 

Entered 

Entered 


Estimate  nDF 

2.76944412  1 

0  1 

0  1 

0  1 

0  1 

0.26900525  1 

0  1 

0.43864757  1 

0  1 

1.37368619  1 

0  1 

0  1 

0  1 

0  1 

-0.7585012  1 

0.93677846  1 

0  1 

0  1 

-0.6638339  1 

0  1 

-0.2663608  1 

0.38056655  1 

0.25611918  1 

0  1 

0  1 

0  1 

-0.3856916  1 

0  1 

0  1 

0  1 

0.38521091  1 

0  1 

-0.2840337  1 

0  1 

0  1 

-0.2081899  1 

-0.2070226  1 

0  1 


SS 

0 

2.000107 

1.220181 

0.14392 

0.222457 

5.654029 

0.724546 

8.338643 

0.014324 

30.66285 

0.80781 

2.12537 

0.014941 

0.56201 

7.867534 

17.76188 

0.11023 

1.272751 

17.36173 

0.096404 

4.293094 

10.58379 

6.324151 

1.606231 

0.132083 

0.976284 

10.33005 

0.539967 

0.482103 

1.658614 

8.603814 

0.124385 

6.291083 

0.118856 

0.966159 

3.120533 

3.270184 

0.045916 


"Sig  Prob" 
0.0044 
0.0119 
0.0934 
0.2201 
0.1426 
0.2045 
0.0697 
0.1094 
0.1230 
0.0850 
0.1550 
0.2035 
0.2204 
0.2260 


Seq  SS 
21.88207 
16.17656 
6.997021 
3.704105 
5.258663 
3.908781 
7.893483 
6.048723 
5.544293 
6.813147 
4.585196 
3.640248 
3.364367 
3.270184 


RSquare 

0.0632 

0.1099 

0.1302 

0.1409 

0.1561 

0.1673 

0.1901 

0.2076 

0.2236 

0.2433 

0.2566 

0.2671 

0.2768 

0.2862 


"F  Ratio"  "Prob>F" 

0.000  1.0000 

0.906  0.3433 

0.551  0.4595 

0.065  0.7997 

0.100  0.7524 

2.563  0.1122 

0.326  0.5689 

3.780  0.0544 

0.006  0.9362 

13.900  0.0003 

0.364  0.5475 

0.963  0.3285 

0.007  0.9348 

0.253  0.6159 

3.566  0.0615 

8.052  0.0054 

0.050  0.8243 

0.575  0.4500 

7.870  0.0059 

0.043  0.8355 

1.946  0.1658 

4.798  0.0306 

2.867  0.0932 

0.726  0.3959 

0.059  0.8079 

0.440  0.5083 

4.683  0.0326 

0.243  0.6229 

0.217  0.6422 

0.750  0.3883 

3.900  0.0507 

0.056  0.8135 

2.852  0.0941 

0.053  0.8176 

0.436  0.5105 

1.415  0.2368 

1.482  0.2260 

0.021  0.8860 


Cp  p 

5.3448  2 

0.9423  3 

0.1729  4 

0.7069  5 

0.6256  6 

1.0785  7 

-0.046  8 

-0.44  9 

-0.634  10 

-1.331  11 

-1.145  12 

-0.586  13 

0.0823  14 

0.788  15 


Appendix  D 


Response  Column  59 
Actual  by  Predicted  Plot 


Summary  of  Fit 


RSquare 
RSquare  Adj 
Root  Mean  Square  Error 
Mean  of  Response 
Observations  (or  Sum  Wgts) 
Analysis  of  Variance 
Source  DF 

Model  14 

Error  112 

C.  Total  126 

Lack  Of  Fit 

Source  DF 

Lack  Of  Fit  72 

Pure  Error  40 

Total  Error  112 


0.286248 

0.197029 

1.485257 

3.850394 
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Sum  of  Squares 
99.08684 
247.07064 
346.15748 


Sum  of  Squares 
201.15397 
45.91667 
247.07064 


Parameter  Estimates 

Term 

Intercept 

whattypeshrinkbusiness[1  -0] 
whattypeshrinkinternet[1-0] 
whattypecomponentCASE[1  -0] 
whattypeenterprisemanufact[1  -0] 
whattypeenterprisepayroll[1  -0] 
whattypeenterpriseweb[1-0] 
whatprocessdesign[1-0] 
whatprocesstesting[1  -0] 
whatprocessmaintenance[1  -0] 
whatprocessspecification[1-0] 
whatprocessCM[1  -0] 
whatprocessSWEngSuppt[1  -0] 
whatproductsCOTS[1-0] 
whatproductscommoncust[1  -0] 


Effect  Tests 

Source  Nparm 

whattypeshrinkbusiness  1 

whattypeshrinkinternet  1 

whattypecomponentCASE  1 

whattypeenterprisemanufact  1 

whattypeenterprisepayroll  1 

whattypeenterpriseweb  1 

whatprocessdesign  1 

whatprocesstesting  1 

whatprocessmaintenance  1 

whatprocessspecification  1 

whatprocessCM  1 

whatprocessSWEngSuppt  1 

whatproductsCOTS  1 

whatproductscommoncust  1 


Mean  Square 
7.07763 
2.20599 


F  Ratio 
3.2084 
Prob  >  F 
0.0003 


Mean  Square  F  Ratio 

2.79381  2.4338 

1.14792  Prob  >  F 

0.0014 
Max  RSq 
0.8674 


Estimate 
4.0358244 
-0.538011 
-0.877295 
-2.747372 
1.5170024 
-1.873557 
1 .3276678 
0.5327216 
-0.761133 
-0.512238 
0.7713833 
-0.770422 
0.5680674 
0.4163799 
0.4140452 


Std  Error 
0.325364 
0.336057 
0.451232 
0.736907 
0.803283 
0.660274 
0.473254 
0.381871 
0.347489 
0.302533 
0.356468 
0.390108 
0.336387 
0.350088 
0.340066 

Sum  of  Squares 
5.654029 
8.338643 
30.662847 
7.867534 
17.761882 
17.361732 
4.293094 
10.583792 
6.324151 
10.330046 
8.603814 
6.291083 
3.120533 
3.270184 


t  Ratio 
12.40 
-1.60 
-1.94 
-3.73 
1.89 
-2.84 
2.81 
1.40 
-2.19 
-1.69 
2.16 
-1.97 
1.69 
1.19 
1.22 


Prob>|t| 

<.0001 

0.1122 

0.0544 

0.0003 

0.0615 

0.0054 

0.0059 

0.1658 

0.0306 

0.0932 

0.0326 

0.0507 

0.0941 

0.2368 

0.2260 


F  Ratio 
2.5630 
3.7800 
13.8998 
3.5664 
8.0517 
7.8703 
1.9461 
4.7978 
2.8668 
4.6827 
3.9002 
2.8518 
1.4146 
1.4824 


Prob  >  F 
0.1122 
0.0544 
0.0003 
0.0615 
0.0054 
0.0059 
0.1658 
0.0306 
0.0932 
0.0326 
0.0507 
0.0941 
0.2368 
0.2260 


Cost  =  4.04  +  (-.54)shrink-bus  +  (-.88)shrink-int  +  (-2.75)comp-case  +  (1.52)ent-mnft  +  (-1 .87)ent-pay  +  (1.33)ent-web  + 
(.53)proc-design  +  (-.76)proc-test  +  (-.51)proc-maint  +  (.77)proc-spec  +  (-.77)proc-cm  +  (.57)proc-swengsup  +  (.42)prod- 
cots  +  (.41)prod-comcust 
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Stepwise  Fit  -  Combined  Survey  Data  -  Consequences  (Sched) 

Response: 

Column  59 


Stepwise  Regression  Control 

Prob  to  Enter  0.250 

Prob  to  Leave  0.250 

Direction: 

Rules: 


Current  Estimates 

SSE  DFE 

212.52433  112 


MSE  RSquare 

1.8975386  0.3073 


RSquare  Adj  Cp  AIC 

0.2207  -1.359694  95.38841 


Lock 

X 


Entered  Parameter 

X  Intercept 

X  whattypesystemsavionics{0-1} 

X  whattypesystemsembedded{0-1} 

whattypesystemscommunications{0-1} 
X  whattypesystemsdevice{0-1} 

X  whattypeshrinkbusiness{0-1} 

whattypeshrinkutilities{0-1 } 
whattypeshrinkinternet{0-1 } 
whattypecomponentdomain{0-1} 
whattypecomponentCASE{0-1 } 
whattypecomponentclass{0- 1 } 
whattypecomponentOS{0-1} 
whattypecomponentdevelopment{0-1} 
whattypeenterpriseacctng{0-1 } 
whattypeenterprisemanufact{0-1 } 

X  whattypeenterprisepayroll{0-1} 

whattypeenterpriseOES{0- 1 } 
whattypeenterprisescripting{0-1} 

X  whattypeenterpriseweb{0-1} 

X  whatprocessrequirements{0-1} 

X  whatprocessdesign{0-1} 

whatprocesstesting{0-1} 
whatprocessmaintenance{0-1} 

X  whatprocessreengineering{0-1} 

whatprocessappsuppt{0-1 } 
whatprocesstraining{0-1} 
whatprocessspecification{0-1 } 

X  whatprocessdocumentation{0-1} 

whatprocesscoding{0- 1 } 

X  whatprocessfielding{0-1} 

X  whatprocessCM{0-1} 

whatprocesstoolsuppt{0-1} 

X  whatprocessSWEngSuppt{0-1} 

whatprocessnone{0-1 } 
whatproductscustom{0-1 } 
whatproductsCOTS{0-1} 

X  whatproductscommoncust{0-1} 

whatproductsnone{0-1 } 


Step  History 

Step  Parameter 

1  whatprocessdesign{0-1} 

2  whatprocessCM{0-1} 

3  whatprocessfielding{0-1} 

4  whattypeenterpriseweb{0-1} 

5  whattypeshrinkbusiness{0-1} 

6  whattypesystemsavionics{0-1} 

7  whatproductscommoncust{0-1} 

8  whatprocessreengineering{0-1} 

9  whatprocessrequirements{0-1} 

10  whatprocessSWEngSuppt{0-1} 

1 1  whattypesystemsembedded{0-1 } 

12  whattypesystemsdevice{0-1} 

13  whattypeenterprisepayroll{0-1} 

14  whatprocessdocumentation{0-1} 


Action 

Entered 

Entered 

Entered 

Entered 

Entered 

Entered 

Entered 

Entered 

Entered 

Entered 

Entered 

Entered 

Entered 

Entered 


Estimate  nDF 

4.64369106  1 

-0.5854713  1 

-0.3376822  1 

0  1 

0.43635426  1 

-0.3042691  1 

0  1 

0  1 

0  1 

0  1 

0  1 

0  1 

0  1 

0  1 

0  1 

0.40849613  1 

0  1 

0  1 

-0.5260352  1 

0.40987844  1 

-0.5674255  1 

0  1 

0  1 

-0.408045  1 

0  1 

0  1 

0  1 

0.27611673  1 

0  1 

-0.5713509  1 

0.72374226  1 

0  1 

-0.2900281  1 

0  1 

0  1 

0  1 

-0.2550195  1 

0  1 


"Sig  Prob" 
0.0301 
0.0315 
0.0048 
0.0291 
0.1177 
0.1151 
0.1088 
0.1413 
0.0592 
0.2083 
0.1655 
0.1765 
0.1888 
0.2110 


Seq  SS 
11.37862 
10.86486 
17.89577 
10.24738 
5.155383 
5.167206 
5.283874 
4.394829 
7.11394 
3.121599 
3.767646 
3.551512 
3.333608 
3.002598 


SS 

0 

7.788231 
5.134244 
0.393926 
4.886104 
6.741254 
0.058719 
0.897267 
0.799475 
1.442429 
0.530359 
0.00018 
1.356874 
1 .305325 
0.000904 
3.354843 
0.251672 
0.817272 
10.68002 
10.73831 
22.81655 
0.070548 
0.703141 
9.497281 
0.11362 
0.372171 
0.030108 
3.002598 
0.67431 
13.03802 
29.67625 
0.113074 
5.670811 
0.905168 
0.031603 
0.99783 
4.647021 
0.285312 


RSquare 

0.0371 

0.0725 

0.1308 

0.1642 

0.1810 

0.1979 

0.2151 

0.2294 

0.2526 

0.2628 

0.2751 

0.2866 

0.2975 

0.3073 


"F  Ratio"  "Prob>F" 

0.000  1.0000 

4.104  0.0451 

2.706  0.1028 

0.206  0.6507 

2.575  0.1114 

3.553  0.0620 

0.031  0.8613 

0.471  0.4941 

0.419  0.5187 

0.759  0.3857 

0.278  0.5993 

0.000  0.9923 

0.713  0.4002 

0.686  0.4093 

0.000  0.9827 

1.768  0.1863 

0.132  0.7175 

0.429  0.5141 

5.628  0.0194 

5.659  0.0191 

12.024  0.0007 

0.037  0.8481 

0.368  0.5451 

5.005  0.0273 

0.059  0.8079 

0.195  0.6599 

0.016  0.9004 

1.582  0.2110 

0.353  0.5535 

6.871  0.0100 

15.639  0.0001 

0.059  0.8084 

2.989  0.0866 

0.475  0.4922 

0.017  0.8980 

0.524  0.4708 

2.449  0.1204 

0.149  0.7000 


Cp  p 

9.9471  2 

7.0577  3 

1.0042  4 

-1.607  5 

-1.927  6 

-2.253  7 

-2.631  8 

-2.608  9 

-3.81  10 

-3.214  11 

-2.91  12 

-2.508  13 

-2.008  14 

-1.36  15 


Appendix  D 


Response  Column  59 
Actual  by  Predicted  Plot 


1  2  3  4  5  6  7 

Column  59  Predicted  P<.0001  RSq=0.31 
RMSE=1.3775 


Summary  of  Fit 


RSquare 
RSquare  Adj 
Root  Mean  Square  Error 
Mean  of  Response 
Observations  (or  Sum  Wgts) 
Analysis  of  Variance 
Source  DF 

Model  14 

Error  112 

C.  Total  126 

Lack  Of  Fit 

Source  DF 

Lack  Of  Fit  64 

Pure  Error  48 

Total  Error  112 


0.307294 

0.220706 

1.377512 

4.03937 
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Sum  of  Squares 
94.27882 
212.52433 
306.80315 


Mean  Square  F  Ratio 

6.73420  3.5489 

1 .89754  Prob  >  F 

<.0001 


Sum  of  Squares 
129.87909 
82.64524 
212.52433 


Mean  Square  F  Ratio 

2.02936  1.1786 

1.72178  Prob  >  F 

0.2775 
Max  RSq 
0.7306 


Parameter  Estimates 

Term 

Intercept 

whattypesystemsavionics[1  -0] 
whattypesystemsembedded  [1  -0] 
whattypesystemsdevice[1  -0] 
whattypeshrinkbusiness[1  -0] 
whattypeenterprisepayroll[1  -0] 
whattypeenterpriseweb[1  -0] 
whatprocessrequirementsjl  -0] 
whatprocessdesignfl  -0] 
whatprocessreengineering[1-0] 
whatprocessdocumentationjl  -0] 
whatprocessfieldingjl  -0] 
whatprocessCM[1  -0] 
whatprocessSWEngSuppt[1  -0] 
whatproductscommoncustfl  -0] 
Effect  Tests 


Estimate  Std  Error  t  Ratio 


3.0529519 

1.1709425 

0.6753645 

-0.872709 

0.6085381 

-0.816992 

1.0520704 

-0.819757 

1.134851 

0.8160901 

-0.552233 

1.1427018 

-1.447485 

0.5800562 

0.5100391 


0.293744 

0.577978 

0.410578 

0.543855 

0.322859 

0.614437 

0.44346 

0.344598 

0.327272 

0.364782 

0.439005 

0.435936 

0.36602 

0.335539 

0.32592 


10.39 

2.03 

1.64 

-1.60 


-1.33 

2.37 

-2.38 

3.47 

2.24 

-1.26 

2.62 

-3.95 

1.73 

1.56 


Prob>|t| 

<.0001 

0.0451 

0.1028 

0.1114 

0.0620 

0.1863 

0.0194 

0.0191 

0.0007 

0.0273 

0.2110 

0.0100 

0.0001 

0.0866 

0.1204 


Source 

whattypesystemsavionics 

whattypesystemsembedded 

whattypesystemsdevice 

whattypeshrinkbusiness 

whattypeenterprisepayroll 

whattypeenterpriseweb 

whatprocessrequirements 

whatprocessdesign 

whatprocessreengineering 

whatprocessdocumentation 

whatprocessfielding 

whatprocessCM 

whatprocessSWEngSuppt 

whatproductscommoncust 


Nparm  DF 

1  1 

1  1 

1  1 

1  1 

1  1 

1  1 

1  1 

1  1 

1  1 

1  1 

1  1 

1  1 

1  1 

1  1 


Sum  of  Squares 
7.788231 
5.134244 
4.886104 
6.741254 
3.354843 
10.680021 
10.738307 
22.816553 
9.497281 
3.002598 
13.038015 
29.676252 
5.670811 
4.647021 


F  Ratio  Prob  >  F 

4.1044  0.0451 

2.7057  0.1028 

2.5750  0.1114 

3.5526  0.0620 

1.7680  0.1863 

5.6284  0.0194 

5.6591  0.0191 

12.0243  0.0007 

5.0051  0.0273 

1.5824  0.2110 

6.8710  0.0100 

15.6393  0.0001 

2.9885  0.0866 

2.4490  0.1204 


Sched  =  3.05  +  (1.17)sys-avia  +  (.68)sys-embed  +  (-.87)sys-dev  +  (.61)shrink-bus  +  (-.82)ent-pay  +  (1.05)ent-web  +  (- 
.82)proc-req  +  (1.13)proc-design  +  (.82)proc-reeng  +  (-.55)proc-doc  +  (1.14)proc-field  +  (-1 .45)proc-cm  +  (.58)proc- 
swengsup  +  (.51)prod-comcust 
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Stepwise  Fit  -  Combined  Survey  Data  -  Consequences  (IntelCapital) 

Response: 

Column  59 


Stepwise  Regression  Control 

Prob  to  Enter  0.250 

Prob  to  Leave  0.250 

Direction: 

Rules: 


Current  Estimates 

SSE  DFE 

94.202467  110 


MSE  RSquare 

0.8563861  0.3778 


RSquare  Adj  Cp 

0.2986  -1.163116 


AIC 

-5.35842 


Lock 

X 


X  Intercept 

whattypesystemsavionics{0-1} 
whattypesystemsembedded{0-1 } 
whattypesystemscommunications{0-1} 
whattypesystemsdevice{0-1} 
whattypeshrinkbusiness{0-1} 
whattypeshrinkutilities{0-1 } 

X  whattypeshrinkinternet{0-1} 

X  whattypecomponentdomain{0-1} 

whattypecomponentCASE{0-1} 
whattypecomponentclass{0- 1 } 
whattypecomponentOS{0-1} 

X  whattypecomponentdevelopment{0-1} 

whattypeenterpriseacctng{0-1 } 
whattypeenterprisemanufact{0-1 } 
whattypeenterprisepayroll{0-1 } 
whattypeenterpriseOES{0- 1 } 
whattypeenterprisescripting{0-1} 

X  whattypeenterpriseweb{0-1} 

whatprocessrequirements{0-1} 
whatprocessdesign{0- 1 } 

X  whatprocesstesting{0-1} 

whatprocessmaintenance{0-1} 

X  whatprocessreengineering{0-1} 

X  whatprocessappsuppt{0-1} 

X  whatprocesstraining{0-1} 

X  whatprocessspecification{0-1} 

whatprocessdocumentation{0-1 } 

X  whatprocesscoding{0-1} 

X  whatprocessfielding{0-1} 

whatprocessCM{0-1} 
whatprocesstoolsuppt{0-1} 
whatprocessSWEngSuppt{0- 1 } 

X  whatprocessnone{0-1} 

whatproductscustom{0-1 } 

X  whatproductsCOTS{0-1} 

X  whatproductscommoncust{0-1} 

whatproductsnone{0-1 } 


Step  History 

Step  Parameter 

1  whatprocesstraining{0-1} 

2  whatprocessfielding{0-1} 

3  whattypecomponentdomain{0-1} 

4  whatprocessappsuppt{0-1} 

5  whatprocesstesting{0-1} 

6  whatprocessnone{0-1} 

7  whatproductsCOTS{0-1} 

8  whattypecomponentdevelopment{0- 1 } 

9  whatprocessreengineering{0-1} 

10  whatprocessspecification{0-1} 

11  whattypeshrinkinternet{0-1} 

12  whattypeenterpriseweb{0-1} 

13  whatproductscommoncust{0-1} 

14  whatprocesscoding{0-1} 


Action 

Entered 

Entered 

Entered 

Entered 

Entered 

Entered 

Entered 

Entered 

Entered 

Entered 

Entered 

Entered 

Entered 

Entered 


Estimate  nDF 

4.05178398  1 

0  1 

0  1 

0  1 

0  1 

0  1 

0  1 

-0.3137522  1 

-0.5406269  1 

0  1 

0  1 

0  1 

0.25366082  1 

0  1 

0  1 

0  1 

0  1 

0  1 

0.27963514  1 

0  1 

0  1 

0.18111677  1 

0  1 

-0.2966347  1 

-0.3369586  1 

-0.4060855  1 

0.23032763  1 

0  1 

-0.183479  1 

0.50022519  1 

0  1 

0  1 

0  1 

0.51766013  1 

0  1 

0.27459974  1 

-0.176152  1 

0  1 


SS 

0 

0.237842 

0.733042 

0.012341 

0.286251 

1.085852 

0.16679 

4.087482 

8.089787 

0.152044 

0.000981 

0.923137 

4.198426 

0.050445 

0.031476 

1.091217 

0.4502 

0.304586 

2.655465 

0.196858 

0.074975 

2.765219 

0.143675 

5.425516 

6.544399 

9.045147 

3.81829 

0.000102 

1.911902 

9.883859 

0.158983 

0.173466 

0.153386 

1.923734 

0.136109 

5.020974 

2.31433 

0.02695 


"F  Ratio" 
0.000 
0.276 
0.855 
0.014 
0.332 
1.271 
0.193 
4.773 
9.446 
0.176 
0.001 
1.079 
4.902 
0.058 
0.036 
1.277 
0.523 
0.354 
3.101 
0.228 
0.087 
3.229 
0.166 
6.335 
7.642 
10.562 
4.459 
0.000 
2.233 
11.541 
0.184 
0.201 
0.178 
2.246 
0.158 
5.863 
2.702 
0.031 


'Prob>F" 

1.0000 

0.6005 

0.3572 

0.9051 

0.5655 

0.2620 

0.6610 

0.0310 

0.0027 

0.6755 

0.9732 

0.3013 

0.0289 

0.8095 

0.8490 

0.2609 

0.4709 

0.5533 

0.0810 

0.6338 

0.7688 

0.0751 

0.6840 

0.0133 

0.0067 

0.0015 

0.0370 

0.9913 

0.1380 

0.0009 

0.6686 

0.6547 

0.6741 

0.1368 

0.6920 

0.0171 

0.1031 

0.8601 


'Sig  Prob"  Seq  SS 

0.0041  9.831898 

0.0029  9.967993 

0.0107  6.918704 

0.0407  4.292872 

0.0675  3.34841 

0.0786  3.040774 

0.1037  2.56198 

0.1444  2.036393 

0.1010  2.539941 

0.0834  2.784603 

0.0843  2.720306 

0.0634  3.085385 

0.1181  2.148373 

0.1380  1.911902 


RSquare  Cp  p 

0.0649  20.011  2 

0.1308  12.081  3 

0.1765  7.1896  4 

0.2048  4.9134  5 

0.2270  3.5779  6 

0.2470  2.549  7 

0.2640  1.9969  8 

0.2774  1.9684  9 

0.2942  1.4384  10 

0.3126  0.6646  11 

0.3306  -0.045  12 

0.3509  -1.119  13 

0.3651  -1.259  14 

0.3778  -1.163  15 
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Response  Column  59 
Actual  by  Predicted  Plot 


1  2  3  4  5  6  7 

Column  59  Predicted  P<.0001  RSq=0.38 
RMSE=0.9254 


Summary  of  Fit 


RSquare 
RSquare  Adj 
Root  Mean  Square  Error 
Mean  of  Response 
Observations  (or  Sum  Wgts) 
Analysis  of  Variance 
Source  DF 

Model  14 

Error  110 

C.  Total  124 

Lack  Of  Fit 

Source  DF 

Lack  Of  Fit  62 

Pure  Error  48 

Total  Error  110 


0.377758 

0.298564 

0.925411 

4.192 

125 


Sum  of  Squares 
57.18953 
94.20247 
151.39200 


Sum  of  Squares 
57.271154 
36.931313 
94.202467 


Mean  Square 
4.08497 
0.85639 


Mean  Square 
0.923728 
0.769402 


Parameter  Estimates 

Term 

Intercept 

whattypeshrinkinternet[1-0] 
whattypecomponentdomain[1  -0] 
whattypecomponentdevelopment[1-0] 
whattypeenterpriseweb[1  -0] 
whatprocesstesting[1  -0] 
whatprocessreengineering[1-0] 
whatprocessappsuppt[1-0] 
whatprocesstraining[1  -0] 
whatprocessspecification[1  -0] 
whatprocesscoding[1  -0] 
whatprocessfieldingjl  -0] 
whatprocessnone[1  -0] 
whatproductsCOTS[1-0] 
whatproductscommoncust[1  -0] 
Effect  Tests 
Source 

whattypeshrinkinternet 

whattypecomponentdomain 

whattypecomponentdevelopment 

whattypeenterpriseweb 

whatprocesstesting 

whatprocessreengineering 

whatprocessappsuppt 

whatprocesstraining 

whatprocessspecification 

whatprocesscoding 

wh  atprocessfield  i  ng 

whatprocessnone 

wh  atprod  uctsCOT  S 

whatproductscommoncust 


Estimate 

4.0353203 

0.6275045 

1.0812539 

-0.507322 

-0.55927 

-0.362234 

0.5932694 

0.6739173 

0.8121711 

-0.460655 

0.3669581 

-1.00045 

-1.03532 

-0.549199 

0.3523041 


Nparm  DF 

1  1 

1  1 

1  1 

1  1 

1  1 

1  1 

1  1 

1  1 

1  1 

1  1 

1  1 

1  1 

1  1 

1  1 


F  Ratio 
4.7700 
Prob  >  F 
<.0001 


F  Ratio 
1.2006 
Prob  >  F 
0.2563 
Max  RSq 
0.7561 


Std  Error 
0.221309 
0.287226 
0.351798 
0.229126 
0.317604 
0.201585 
0.235704 
0.243785 
0.249905 
0.218161 
0.245594 
0.294488 
0.690776 
0.226815 
0.214309 


Sum  of  Squares 
4.0874823 
8.0897873 
4.1984259 
2.6554655 
2.7652186 
5.4255160 
6.5443987 
9.0451466 
3.8182902 
1.9119024 
9.8838592 
1.9237345 
5.0209744 
2.3143301 


t  Ratio  Prob>|t| 

18.23  <.0001 

2.18  0.0310 

3.07  0.0027 

-2.21  0.0289 

-1.76  0.0810 

-1.80  0.0751 

2.52  0.0133 

2.76  0.0067 

3.25  0.0015 

-2.11  0.0370 

1.49  0.1380 

-3.40  0.0009 

-1.50  0.1368 

-2.42  0.0171 

1.64  0.1031 


F  Ratio  Prob  >  F 

4.7729  0.0310 

9.4464  0.0027 

4.9025  0.0289 

3.1008  0.0810 

3.2289  0.0751 

6.3354  0.0133 

7.6419  0.0067 

10.5620  0.0015 

4.4586  0.0370 

2.2325  0.1380 

11.5414  0.0009 

2.2463  0.1368 

5.8630  0.0171 

2.7024  0.1031 


IntelCap  =  4.04  +  (.63)shrink-int  +  (1.08)comp-domain  +  (-.51)comp-dev  +  (-.56)ent-web  +  (-.36)proc-test  +  (.59)proc- 
reeng  +  (.67)proc-appsup  +  (.81)proc-train  +  (-.46)proc-spec  +  (.37)proc-coding  +  (-1 .00)proc-field  +  (-1 .04)proc-none  +  (- 
.55)prod-cots  +  (.35)prod-comcust 
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Stepwise  Fit  -  Combined  Survey  Data  -  Consequences  (SchedFlex) 

Response: 

Column  59 

Stepwise  Regression  Control 

Prob  to  Enter  0.250 

Prob  to  Leave  0.250 

Direction: 

Rules: 


Current  Estimates 

SSE  DFE 

222.71682  115 


MSE  RSquare 

1.936668  0.2128 


RSquare  Adj 
0.1444 


Cp  AIC 

-1.768193  93.77202 


Lock 

X 


Entered  Parameter 

X  Intercept 

X  whattypesystemsavionics{0-1} 

X  whattypesystemsembedded{0-1} 

X  whattypesystemscommunications{0-1 } 

whattypesystemsdevice{0- 1 } 

X  whattypeshrinkbusiness{0-1} 

whattypeshrinkutilities{0-1} 

X  whattypeshrinkinternet{0-1} 

X  whattypecomponentdomain{0-1} 

X  whattypecomponentCASE{0-1} 

whattypecomponentclass{0- 1 } 

X  whattypecomponentOS{0-1} 

whattypecomponentdevelopment{0-1} 
X  whattypeenterpriseacctng{0-1} 

whattypeenterprisemanufact{0-1 } 
whattypeenterprisepayroll{0-1 } 
whattypeenterpriseOES{0- 1 } 
whattypeenterprisescripting{0- 1 } 
whattypeenterpriseweb{0-1 } 
whatprocessrequirements{0-1} 
whatprocessdesign{0- 1 } 
whatprocesstesting{0- 1 } 
whatprocessmaintenance{0-1} 
whatprocessreengineering{0-1} 
whatprocessappsuppt{0-1 } 
whatprocesstraining{0-1} 
whatprocessspecification{0-1 } 
whatprocessdocumentation{0-1 } 
whatprocesscoding{0- 1 } 

X  whatprocessfielding{0-1} 
whatprocessCM{0-1} 
whatprocesstoolsuppt{0-1} 
whatprocessSWEngSuppt{0- 1 } 
whatprocessnone{0-1 } 
whatproductscustom{0-1 } 
whatproductsCOTS{0-1} 
whatproductscommoncust{0-1} 
whatproductsnone{0-1 } 


Step  History 

Step  Parameter 

1  whatprocesstoolsuppt{0-1} 

2  whattypesystemsembedded{0-1} 

3  whattypecomponentCASE{0-1} 

4  whattypesystemsavionics{0-1} 

5  whatprocessfielding{0-1} 

6  whatprocesstoolsuppt{0-1} 

7  whattypeshrinkbusiness{0-1} 

8  whattypesystemscommunications{0-1 } 

9  whattypeenterpriseacctng{0-1} 

10  whattypecomponentdomain{0-1} 

11  whattypeshrinkinternet{0-1} 

12  whattypecomponentOS{0-1} 


Estimate  nDF 

2.5772971 1  1 

0.50174885  1 

0.46735013  1 

-0.3280729  1 

0  1 

0.21223729  1 

0  1 

-0.2741296  1 

0.40947514  1 

0.57154378  1 

0  1 

0.33339491  1 

0  1 

-0.331847  1 

0  1 

0  1 

0  1 

0  1 

0  1 

0  1 

0  1 

0  1 

0  1 

0  1 

0  1 

0  1 

0  1 

0  1 

0  1 

0.43332207  1 

0  1 

0  1 

0  1 

0  1 

0  1 

0  1 

0  1 

0  1 


SS 

0 

5.921546 

10.15151 

8.231976 

0.047589 

3.451627 

1.240597 

3.595605 

4.282942 

5.769236 

1.676531 

3.06607 

1.54347 

3.833803 

0.690629 

0.926397 

2.336194 

1.286342 

0.162797 

0.419883 

0.000094 

1.728722 

0.06462 

0.180816 

0.420659 

0.176785 

0.692242 

1.08393 

1.009188 

10.14757 

0.015701 

0.310249 

0.936591 

0.270829 

0.727528 

0.004639 

0.941994 

2.256109 


"F  Ratio" 
0.000 
3.058 
5.242 
4.251 
0.024 
1.782 
0.639 
1.857 
2.212 

2.979 
0.865 
1.583 
0.796 

1.980 
0.355 
0.476 
1.208 
0.662 
0.083 
0.215 
0.000 
0.892 
0.033 
0.093 
0.216 
0.091 
0.355 
0.558 
0.519 
5.240 
0.008 
0.159 
0.481 
0.139 
0.374 
0.002 
0.484 
1.167 


'Prob>F" 

1.0000 

0.0830 

0.0239 

0.0415 

0.8762 

0.1845 

0.4259 

0.1757 

0.1397 

0.0870 

0.3544 

0.2109 

0.3743 

0.1621 

0.5527 

0.4916 

0.2739 

0.4175 

0.7733 

0.6435 

0.9945 

0.3470 

0.8560 

0.7614 

0.6432 

0.7640 

0.5522 

0.4568 

0.4728 

0.0239 

0.9287 

0.6908 

0.4892 

0.7102 

0.5423 

0.9612 

0.4879 

0.2824 


Action 

Entered 

Entered 

Entered 

Entered 

Entered 

Removed 

Entered 

Entered 

Entered 

Entered 

Entered 

Entered 


"Sig  Prob"  Seq  SS 

0.0378  9.71645 

0.0326  9.999263 

0.0905  6.135493 

0.0907  6.032095 

0.0986  5.666724 

0.2944  2.267766 

0.1014  5.498391 

0.1190  4.916622 

0.1172  4.904046 

0.1716  3.696445 

0.2289  2.847915 

0.2109  3.06607 


RSquare  Cp  p 

0.0343  3.4102  2 

0.0697  0.8204  3 

0.0914  0.004  4 

0.1127  -0.765  5 

0.1327  -1.366  6 

0.1247  -2.325  5 

0.1441  -2.849  6 

0.1615  -3.106  7 

0.1788  -3.357  8 

0.1919  -3.054  9 

0.2020  -2.361  10 

0.2128  -1.768  11 


Appendix  D 


Response  Column  59 


Actual  by  Predicted  Plot 


7-  ■  ■ 


1-  u  u  m  m 


1  '  i  '  i  t  i  i  i  '  i  ' 

1  2  3  4  5  6  7 

Column  59  Predicted  P=0.0015  RSq=0.21 
RMSE=1 .3916 


Summary  of  Fit 


RSquare  0.212816 

RSquare  Adj  0.144365 

Root  Mean  Square  Error  1.391642 

Mean  of  Response  4.309524 

Observations  (or  Sum  Wgts)  126 


Analysis  of  Variance 

Source 

DF 

Sum  of  Squares 

Mean  Square 

Model 

10 

60.21175 

6.02117 

Error 

115 

222.71682 

1 .93667 

C.  Total 

125 

282.92857 

F  Ratio 
3.1090 
Prob  >  F 
0.0015 


Lack  Of  Fit 


Source  DF 

Lack  Of  Fit  25 

Pure  Error  90 

Total  Error  115 


Sum  of  Squares  Mean  Square 

49.49456  1.97978 

173.22226  1.92469 

222.71682 


F  Ratio 
1.0286 
Prob  >  F 
0.4410 
Max  RSq 
0.3878 


Parameter  Estimates 

Term 

Intercept 

whattypesystemsavionics[1  -0] 
whattypesystemsembeddedjl  -0] 
whattypesystemscommunications[1-0] 
whattypeshrinkbusiness[1  -0] 
whattypeshrinkinternet[1  -0] 
whattypecomponentdomain[1  -0] 
whattypecomponentCASE[1  -0] 
whattypecomponentOSjl  -0] 
whattypeenterpriseacctng[1  -0] 
whatprocessfieldingjl  -0] 


Estimate 

4.5723198 

-1.003498 

-0.9347 

0.6561458 

-0.424475 

0.5482592 

-0.81895 

-1.143088 

-0.66679 

0.6636939 

-0.866644 


Std  Error 
0.20312 
0.573887 
0.408258 
0.318256 
0.317956 
0.402372 
0.550699 
0.66229 
0.529939 
0.471716 
0.378606 


t  Ratio 
22.51 
-1.75 
-2.29 
2.06 
-1.34 
1.36 
-1.49 
-1.73 
-1.26 
1.41 
-2.29 


Prob>|t| 

<.0001 

0.0830 

0.0239 

0.0415 

0.1845 

0.1757 

0.1397 

0.0870 

0.2109 

0.1621 

0.0239 


Effect  Tests 

Source 

whattypesystemsavionics 

whattypesystemsembedded 

whattypesystemscommunications 

whattypeshrinkbusiness 

whattypeshrinkinternet 

whattypecomponentdomain 

whattypecomponentCASE 

whattypecomponentOS 

whattypeenterpriseacctng 

whatprocessfielding 


Nparm  DF 

1  1 

1  1 

1  1 

1  1 

1  1 

1  1 

1  1 

1  1 

1  1 

1  1 


Sum  of  Squares 
5.921546 
10.151508 
8.231976 
3.451627 
3.595605 
4.282942 
5.769236 
3.066070 
3.833803 
10.147570 


F  Ratio  Prob  >  F 

3.0576  0.0830 

5.2417  0.0239 

4.2506  0.0415 

1.7823  0.1845 

1.8566  0.1757 

2.2115  0.1397 

2.9789  0.0870 

1.5832  0.2109 

1.9796  0.1621 

5.2397  0.0239 


SchedFlex  =  4.57  +  sys-avia(-1 .00)  +  sys-embed(-0.93)  +  sys-comm  (0.66)  +  shrink-bus(-0.42)  +  shrink-int(0.55)  +  comp- 
domain(-0.82)  +  comp-CASE(-1.14)  +  comp-OS(-0.67)  +  ent-acct(0.66)  +  proc-field(-0.87) 
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Stepwise  Fit  -  Combined  Survey  Data  -  Consequences  (AdminOverhead) 

Response: 

Column  59 

Stepwise  Regression  Control 

Prob  to  Enter  0.250 

Prob  to  Leave  0.250 

Direction: 

Rules: 


Current  Estimates 

SSE  DFE 

202.49599  115 


MSE  RSquare 

1.7608347  0.3116 


RSquare  Adj 
0.2517 


Cp  AIC 

-6.073093  81.77921 


Entered  Parameter 

X  Intercept 

X  whattypesystemsavionics{0-1} 

whattypesystemsembedded{0-1 } 
whattypesystemscommunications{0-1} 
whattypesystemsdevice{0- 1 } 
whattypeshrinkbusiness{0-1} 

X  whattypeshrinkutilities{0-1} 

whattypeshrinkinternet{0-1 } 
whattypecomponentdomain{0-1} 

X  whattypecomponentCASE{0-1} 

X  whattypecomponentclass{0-1} 

X  whattypecomponentOS{0-1} 

whattypecomponentdevelopment{0-1} 
whattypeenterpriseacctng{0-1} 
whattypeenterprisemanufact{0-1 } 
whattypeenterprisepayroll{0-1} 
whattypeenterpriseOES{0- 1 } 
whattypeenterprisescripting{0-1} 

X  whattypeenterpriseweb{0-1} 

whatprocessrequirements{0-1} 
whatprocessdesign{0- 1 } 
whatprocesstesting{0- 1 } 
whatprocessmaintenance{0-1} 
whatprocessreengineering{0-1} 
whatprocessappsuppt{0-1 } 
whatprocesstraining{0-1} 
whatprocessspecification{0-1 } 
whatprocessdocumentation{0-1 } 
whatprocesscoding{0- 1 } 
whatprocessfielding{0-1} 

X  whatprocessCM{0-1} 

X  whatprocesstoolsuppt{0-1} 

whatprocessSWEngSuppt{0- 1 } 
whatprocessnone{0-1 } 
whatproductscustom{0-1 } 
whatproductsCOTS{0-1} 

X  whatproductscommoncust{0-1} 

X  whatproductsnone{0-1} 


Step  History 

Step  Parameter 


9 

10 


whatproductsnone{0-1 } 
whattypecomponentclass{0-1 } 
whattypecomponentOS{0-1 } 
whatproductscommoncust{0-1} 
whattypesystemsavionics{0-1} 
whatprocessCM{0-1} 
whatprocesstoolsuppt{0-1} 
whattypecomponentCASE{0-1 } 
whattypeenterpriseweb{0-1 } 
whattypeshrinkutilities{0-1 } 


Action 

Entered 

Entered 

Entered 

Entered 

Entered 

Entered 

Entered 

Entered 

Entered 

Entered 


Estimate  nDF 

2.90816156  1 

-0.5780364  1 

0  1 

0  1 

0  1 

0  1 

-0.4503626  1 

0  1 

0  1 

0.64085321  1 

0.87068312  1 

0.47669297  1 

0  1 

0  1 

0  1 

0  1 

0  1 

0  1 

-0.4150102  1 

0  1 

0  1 

0  1 

0  1 

0  1 

0  1 

0  1 

0  1 

0  1 

0  1 

0  1 

0.55046902  1 

-0.4831141  1 

0  1 

0  1 

0  1 

0  1 

-0.3110571  1 

0.99251455  1 


"Sig  Prob" 
0.0000 
0.0401 
0.0810 
0.1063 
0.0935 
0.1052 
0.0660 
0.0748 
0.0645 
0.1433 


Seq  SS 
38.99206 
8.62782 
6.101291 
5.149513 
5.473953 
5.033623 
6.372138 
5.86815 
6.196537 
3.823846 


SS 

0 

8.103662 

0.101448 

0.751293 

0.727113 

0.200264 

3.823846 

1.489453 

1.918166 

7.087136 

13.66001 

6.510401 

0.588073 

0.011775 

0.06238 

0.017523 

0.745357 

0.585938 

6.8019 

0.042485 

0.003608 

0.062216 

0.026595 

0.004232 

1.742402 

0.964662 

I. 358536 
0.667851 
0.456539 
0.377745 
16.86878 

I I . 48985 
0.630917 
1.478767 
0.017478 
0.538713 
7.264591 

24.315 


RSquare 

0.1326 

0.1619 

0.1826 

0.2001 

0.2188 

0.2359 

0.2575 

0.2775 

0.2986 

0.3116 


"F  Ratio" 
0.000 
4.602 
0.057 
0.425 
0.411 
0.113 
2.172 
0.845 
1.090 
4.025 
7.758 
3.697 
0.332 
0.007 
0.035 
0.010 
0.421 
0.331 
3.863 
0.024 
0.002 
0.035 
0.015 
0.002 
0.989 
0.546 
0.770 
0.377 
0.258 
0.213 
9.580 
6.525 
0.356 
0.839 
0.010 
0.304 
4.126 
13.809 


Cp 

1.3869 

-0.786 

-1.736 

-2.226 

-2.874 

-3.308 

-4.389 

-5.227 

-6.224 

-6.073 


"Prob>F" 

1.0000 

0.0340 

0.8115 

0.5160 

0.5228 

0.7375 

0.1433 

0.3600 

0.2986 

0.0472 

0.0063 

0.0570 

0.5656 

0.9352 

0.8517 

0.9211 

0.5177 

0.5663 

0.0518 

0.8774 

0.9641 

0.8519 

0.9028 

0.9612 

0.3220 

0.4616 

0.3821 

0.5403 

0.6128 

0.6453 

0.0025 

0.0119 

0.5518 

0.3617 

0.9212 

0.5824 

0.0445 

0.0003 

P 

2 

3 

4 

5 

6 

7 

8 

9 

10 

11 
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Response  Column  59 
Actual  by  Predicted  Plot 


Column  59  Predicted  P<.0001  RSq=0.31 
RMSE=1 .327 


Summary  of  Fit 


RSquare  0.311554 

RSquare  Adj  0.251689 

Root  Mean  Square  Error  1 .326964 

Mean  of  Response  4.150794 

Observations  (or  Sum  Wgts)  126 


Analysis  of  Variance 


Source  DF 

Model  10 

Error  115 

C.  Total  125 


Sum  of  Squares 
91 .63893 
202.49599 
294.13492 


Mean  Square  F  Ratio 

9.16389  5.2043 

1 .76083  Prob  >  F 

<.0001 


Lack  Of  Fit 


Source  DF 

Lack  Of  Fit  24 

Pure  Error  91 

Total  Error  115 


Sum  of  Squares 
44.45551 
158.04048 
202.49599 


Mean  Square  F  Ratio 

1.85231  1.0666 

1.73671  Prob  >  F 

0.3967 
Max  RSq 
0.4627 


Parameter  Estimates 

Term 

Intercept 

whattypesystemsavionics[1  -0] 
whattypeshrinkutilities[1  -0] 
whattypecomponentCASE[1  -0] 
whattypecomponentclass[1  -0] 
whattypecomponentOSfl  -0] 
whattypeenterpriseweb[1-0] 
whatprocessCM[1  -0] 
whatprocesstoolsuppt[1  -0] 
whatproductscommoncust[1  -0] 
whatproductsnone[1  -0] 


Estimate 

4.201794 

1.1560728 

0.9007251 

-1.281706 

-1.741366 

-0.953386 

0.8300205 

-1.100938 

0.9662282 

0.6221143 

-1.985029 


Std  Error 
0.17116 
0.538895 
0.611225 
0.63887 
0.625207 
0.49582 
0.422312 
0.355697 
0.378252 
0.306284 
0.534182 


t  Ratio  Prob>|t| 

24.55  <.0001 

2.15  0.0340 

1.47  0.1433 

-2.01  0.0472 

-2.79  0.0063 

-1.92  0.0570 

1.97  0.0518 

-3.10  0.0025 

2.55  0.0119 

2.03  0.0445 

-3.72  0.0003 


Effect  Tests 

Source 

whattypesystemsavionics 

whattypeshrinkutilities 

whattypecomponentCASE 

whattypecomponentclass 

whattypecomponentOS 

whattypeenterpriseweb 

whatprocessCM 

whatprocesstoolsuppt 

whatproductscommoncust 

whatproductsnone 


Nparm  DF 

1  1 

1  1 

1  1 

1  1 

1  1 

1  1 

1  1 

1  1 

1  1 

1  1 


Sum  of  Squares 
8.103662 
3.823846 
7.087136 
13.660013 
6.510401 
6.801900 
16.868779 
1 1 .489854 
7.264591 
24.315003 


F  Ratio  Prob  >  F 

4.6022  0.0340 

2.1716  0.1433 

4.0249  0.0472 

7.7577  0.0063 

3.6973  0.0570 

3.8629  0.0518 

9.5800  0.0025 

6.5252  0.0119 

4.1257  0.0445 

13.8088  0.0003 


AdminOverhead  =  4.20  +  sys-avia(1.16)  +  shrink-util(0.90)  +  comp-CASE(-1.28)  +  comp-class(-1.74)  +  comp-OS(-0.95)  + 
ent-web(0.83)  +  proc-CM(-I.IO)  +  proc-toolsup(0.97)  +  prod-comcust(0.62)  +  prod-none(-1 .99) 
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Stepwise  Fit  -  Combined  Survey  Data  -  Consequences  (ControlProcess) 

Response: 

Column  59 

Stepwise  Regression  Control 

Prob  to  Enter  0.250 

Prob  to  Leave  0.250 

Direction: 

Rules: 


Current  Estimates 

SSE  DFE 

193.74602  114 


MSE  RSquare 

1.6995265  0.3534 


RSquare  Adj 
0.2911 


Cp  AIC 

-4.824553  78.21354 


Lock 

X 


X  Intercept 

whattypesystemsavionics{0-1} 
whattypesystemsembedded{0-1 } 
whattypesystemscommunications{0-1} 
whattypesystemsdevice{0-1} 
whattypeshrinkbusiness{0-1} 
whattypeshrinkutilities{0-1 } 
whattypeshrinkinternet{0-1 } 

X  whattypecomponentdomain{0-1} 

whattypecomponentCASE{0-1} 
whattypecomponentclass{0- 1 } 
whattypecomponentOS{0-1} 
whattypecomponentdevelopment{0-1} 
X  whattypeenterpriseacctng{0-1} 

X  whattypeenterprisemanufact{0-1} 

X  whattypeenterprisepayroll{0-1} 

whattypeenterpriseOES{0- 1 } 
whattypeenterprisescripting{0-1} 
whattypeenterpriseweb{0-1 } 
whatprocessrequirements{0-1} 

X  whatprocessdesign{0-1} 

whatprocesstesting{0-1} 
whatprocessmaintenance{0-1} 

X  whatprocessreengineering{0-1} 

X  whatprocessappsuppt{0-1} 

whatprocesstraining{0-1} 
whatprocessspecification{0-1 } 
whatprocessdocumentation{0-1 } 
whatprocesscoding{0- 1 } 

X  whatprocessfielding{0-1} 

X  whatprocessCM{0-1} 

whatprocesstoolsuppt{0-1} 
whatprocessSWEngSuppt{0- 1 } 

X  whatprocessnone{0-1} 

X  whatproductscustom{0-1} 

whatproductsCOTS{0-1} 
whatproductscommoncust{0-1} 
whatproductsnone{0-1 } 


Step  History 

Step  Parameter 

1  whatprocessreengineering{0-1} 

2  whatprocessdesign{0-1} 

3  whattypeenterprisepayroll{0-1} 

4  whattypeenterpriseacctng{0-1} 

5  whatprocessfielding{0-1} 

6  whatprocessappsuppt{0-1} 

7  whatproductscustom{0-1} 

8  whattypeenterprisemanufact{0-1} 

9  whatprocessnone{0-1} 

10  whattypecomponentdomain{0-1} 

11  whatprocessCM{0-1} 


Action 

Entered 

Entered 

Entered 

Entered 

Entered 

Entered 

Entered 

Entered 

Entered 

Entered 

Entered 


Estimate  nDF 

1.87213301  1 

0  1 

0  1 

0  1 

0  1 

0  1 

0  1 

0  1 

0.36869086  1 

0  1 

0  1 

0  1 

0  1 

-0.3859681  1 

0.60103641  1 

0.85388172  1 

0  1 

0  1 

0  1 

0  1 

0.67690242  1 

0  1 

0  1 

-0.3985334  1 

-0.4309694  1 

0  1 

0  1 

0  1 

0  1 

0.60922  1 

-0.2144393  1 

0  1 

0  1 

0.84344195  1 

-0.2085123  1 

0  1 

0  1 

0  1 


SS 

0 

0.315133 

0.019332 

1.460231 

0.004125 

1.815157 

1.23027 

0.012254 

3.943878 

0.099536 

0.846624 

I. 62914 
0.162198 
5.287453 
5.284862 
14.99363 
1.561557 
0.057367 

0.03425 

0.047585 

35.6875 

I. 372171 
0.125405 

II. 10006 

II. 6104 
0.096731 
0.006109 
0.085356 
0.000086 
16.95242 
3.005997 
1.294476 
0.654743 
5.176113 
4.043738 

0.82184 

0.017826 

0.113113 


"Sig  Prob" 
0.0033 
0.0027 
0.0217 
0.0199 
0.0362 
0.0094 
0.0537 
0.1156 
0.1127 
0.1240 
0.1862 


Seq  SS 
20.23061 
19.78295 
11.01457 
10.9431 
8.565152 
12.679 
6.750676 
4.408897 
4.423842 
4.107921 
3.005997 


RSquare 

0.0675 

0.1335 

0.1703 

0.2068 

0.2354 

0.2777 

0.3002 

0.3149 

0.3297 

0.3434 

0.3534 


"F  Ratio"  "Prob>F" 

0.000  1.0000 

0.184  0.6687 

0.011  0.9156 

0.858  0.3562 

0.002  0.9610 

1.069  0.3035 

0.722  0.3972 

0.007  0.9328 

2.321  0.1304 

0.058  0.8100 

0.496  0.4827 

0.958  0.3297 

0.095  0.7589 

3.111  0.0804 

3.110  0.0805 

8.822  0.0036 

0.918  0.3400 

0.033  0.8552 

0.020  0.8878 

0.028  0.8680 

20.998  0.0000 

0.806  0.3712 

0.073  0.7872 

6.531  0.0119 

6.832  0.0102 

0.056  0.8126 

0.004  0.9525 

0.050  0.8238 

0.000  0.9944 

9.975  0.0020 

1.769  0.1862 

0.760  0.3852 

0.383  0.5372 

3.046  0.0836 

2.379  0.1257 

0.481  0.4892 

0.010  0.9190 

0.066  0.7977 


Cp  p 

18.15  2 

10.228  3 

6.7034  4 

3.2148  5 

0.9188  6 

-3.44  7 

-4.826  8 

-5.038  9 

-5.256  10 

-5.317  11 

-4.825  12 


Appendix  D 


Response  Column  59 
Actual  by  Predicted  Plot 


I 


1  2  3  4  5  6  7 

Column  59  Predicted  P<.0001  RSq=0.35 
RMSE=1 .3037 


Summary  of  Fit 


RSquare 
RSquare  Adj 
Root  Mean  Square  Error 
Mean  of  Response 
Observations  (or  Sum  Wgts) 
Analysis  of  Variance 
Source  DF 

Model  1 1 

Error  114 

C.  Total  125 

Lack  Of  Fit 

Source  DF 

Lack  Of  Fit  37 

Pure  Error  77 

Total  Error  114 


0.353444 

0.291057 

1.303659 

3.674603 

126 


Sum  of  Squares 
105.91271 
193.74602 
299.65873 


Sum  of  Squares 
57.22221 
136.52381 
193.74602 


Parameter  Estimates 

Term 

Intercept 

whattypecomponentdomain[1-0] 
whattypeenterpriseacctng[1-0] 
whattypeenterprisemanufact[1  -0] 
whattypeenterprisepayroll[1  -0] 
whatprocessdesign[1-0] 
whatprocessreengineering[1-0] 
whatprocessappsuppt[1  -0] 
whatprocessfieldingjl  -0] 
whatprocessCM[1  -0] 
whatprocessnone[1  -0] 
whatproductscustom[1  -0] 

Effect  Tests 


Source  Nparm 

whattypecomponentdomain  1 

whattypeenterpriseacctng  1 

whattypeenterprisemanufact  1 

whattypeenterprisepayroll  1 

whatprocessdesign  1 

whatprocessreengineering  1 

whatprocessappsuppt  1 

whatprocessfielding  1 

whatprocessCM  1 

whatprocessnone  1 

whatproductscustom  1 


Mean  Square  F  Ratio 

9.62843  5.6654 

1 .69953  Prob  >  F 

<.0001 


Mean  Square  F  Ratio 

1.54655  0.8723 

1.77304  Prob  >  F 

0.6714 
Max  RSq 
0.5444 


Estimate 
4.1868839 
-0.737382 
0.7719362 
-1.202073 
-1.707763 
-1 .353805 
0.7970667 
0.8619389 
-1.21844 
0.4288786 
-1.686884 
0.4170245 


Std  Error 
0.290782 
0.484055 
0.437645 
0.681676 
0.574961 
0.295435 
0.311886 
0.329774 
0.385791 
0.322481 
0.966601 
0.270355 


t  Ratio 
14.40 
-1.52 
1.76 
-1.76 
-2.97 
-4.58 
2.56 
2.61 
-3.16 
1.33 
-1.75 
1.54 


Prob>|t| 

<.0001 

0.1304 

0.0804 

0.0805 

0.0036 

<.0001 

0.0119 

0.0102 

0.0020 

0.1862 

0.0836 

0.1257 


DF  Sum  of  Squares 

1  3.943878 

1  5.287453 

1  5.284862 

1  14.993628 

1  35.687500 

1  11.100056 

1  11.610399 

1  16.952423 

1  3.005997 

1  5.176113 

1  4.043738 


F  Ratio  Prob  >  F 

2.3206  0.1304 

3.1111  0.0804 

3.1096  0.0805 

8.8222  0.0036 

20.9985  <.0001 

6.5313  0.0119 

6.8315  0.0102 

9.9748  0.0020 

1.7687  0.1862 

3.0456  0.0836 

2.3793  0.1257 


ControlProcess  =  4.19  +  comp-domain(-0.74)  +  ent-acct(0.77)  +  ent-mnft(-1 .20)  +  ent-pay(-1.71)  +  proc-des(-1 .35)  +  proc- 
reeng(0.80)  +  proc-appsup(0.86)  +  proc-field(-1.22)  +  proc-CM(0.43)  +  proc-none(-1 .69)  +  prod-cust(0.42) 
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Stepwise  Fit  -  Combined  Survey  Data  -  Consequences  (InhouseNonCore) 

Response: 

Column  59 

Stepwise  Regression  Control 

Prob  to  Enter  0.250 

Prob  to  Leave  0.250 

Direction: 

Rules: 


Current  Estimates 

SSE  DFE 

177.4713  111 


MSE  RSquare 

1.5988405  0.3192 


RSquare  Adj 
0.2517 


Cp  AIC 

-7.24594  69.09482 


Entered  Parameter 

X  Intercept 

X  whattypesystemsavionics{0-1} 

whattypesystemsembedded{0-1 } 
whattypesystemscommunications{0-1} 
whattypesystemsdevice{0- 1 } 
whattypeshrinkbusiness{0-1} 
whattypeshrinkutilities{0-1 } 

X  whattypeshrinkinternet{0-1} 

whattypecomponentdomain{0-1} 

whattypecomponentCASE{0-1} 

X  whattypecomponentclass{0-1} 

whattypecomponentOS{0-1 } 

X  whattypecomponentdevelopment{0-1} 

X  whattypeenterpriseacctng{0-1} 

whattypeenterprisemanufact{0-1 } 

X  whattypeenterprisepayroll{0-1} 

X  whattypeenterpriseOES{0-1} 

X  whattypeenterprisescripting{0-1} 

X  whattypeenterpriseweb{0-1} 

whatprocessrequirements{0-1} 

X  whatprocessdesign{0-1} 

whatprocesstesting{0-1} 
whatprocessmaintenance{0-1} 
whatprocessreengineering{0-1 } 
whatprocessappsuppt{0-1 } 
whatprocesstraining{0-1} 
whatprocessspecification{0-1 } 
whatprocessdocumentation{0-1 } 
whatprocesscoding{0- 1 } 
whatprocessfielding{0-1} 
whatprocessCM{0-1} 
whatprocesstoolsuppt{0-1} 
whatprocessSWEngSuppt{0- 1 } 
whatprocessnone{0-1 } 
whatproductscustom{0-1 } 
whatproductsCOTS{0-1} 
whatproductscommoncust{0-1} 
whatproductsnone{0-1 } 


Step  History 

Step  Parameter 


9 

10 

11 


whattypeenterpriseweb{0-1 } 

whattypesystemsavionics{0-1 } 

whatproductsnone{0-1} 

whattypecomponentclass{0-1 } 

whattypeshrinkinternet{0-1} 

whattypecomponentdevelopment{0- 1 } 

whattypeenterprisescripting{0-1} 

whattypeenterpriseacctng{0-1} 

whattypeenterpriseOES{0-1} 

whattypeenterprisepayroll{0-1} 

whatprocessdesign{0-1} 


Action 

Entered 

Entered 

Entered 

Entered 

Entered 

Entered 

Entered 

Entered 

Entered 

Entered 

Entered 


Estimate  nDF 


1.30681216 

0.76636033 


0.50519839 

0 

0 

0.96542307 

0 

-0.4635845 

-0.4036303 

0 

-0.5810824 

0.87705647 

0.64992729 

-0.4279849 

0 

-0.1586783 

0 

0 

0 

0 

0 

0 

0 


1 

1 

1 

1 

1 

1 

1 

1 

1 

1 

1 

1 

1 

1 

1 

1 

1 

1 

1 

1 

1 

1 

1 

1 

1 

1 

1 

1 

1 

1 

1 

1 

1 

1 

1 

1 

1 

1 


ss 

0 

14.77664 

0.976616 

1.77829 

0.035784 

0.002858 

0.915975 

11.54484 

0.258895 

0.03722 

11.42311 

0.005453 

12.72151 

4.732332 

1.071863 

5.15367 

5.197706 

9.174103 

6.694961 

0.191404 

2.290248 

0.053223 

0.383084 

0.144334 

0.009056 

0.071521 

0.841749 

0.613843 

0.045271 

0.113927 

0.115151 

0.217301 

1.230072 

0.611462 

0.871021 

0.639741 

0.14805 

7.923065 


"F  Ratio" 
0.000 
9.242 
0.609 
1.113 
0.022 
0.002 
0.571 
7.221 
0.161 
0.023 
7.145 
0.003 
7.957 
2.960 
0.668 
3.223 
3.251 
5.738 
4.187 
0.119 
1.432 
0.033 
0.238 
0.090 
0.006 
0.044 
0.524 
0.382 
0.028 
0.071 
0.071 
0.135 
0.768 
0.380 
0.543 
0.398 
0.092 
4.956 


'Prob>F" 

1.0000 

0.0030 

0.4370 

0.2937 

0.8819 

0.9665 

0.4516 

0.0083 

0.6893 

0.8795 

0.0087 

0.9537 

0.0057 

0.0881 

0.4154 

0.0753 

0.0741 

0.0183 

0.0431 

0.7310 

0.2339 

0.8562 

0.6267 

0.7653 

0.9404 

0.8336 

0.4706 

0.5379 

0.8673 

0.7909 

0.7898 

0.7142 

0.3828 

0.5387 

0.4630 

0.5295 

0.7624 

0.0280 


'Sig  Prob"  Seq  SS 

0.0067  15.43067 

0.0398  8.525146 

0.0255  9.755579 

0.0467  7.513245 

0.0379  7.969779 

0.0197  9.721023 

0.0132  10.53184 

0.1515  3.435105 

0.1338  3.716006 

0.1036  4.322979 

0.2339  2.290248 


RSquare  Cp  p 

0.0592  7.7973  2 

0.0919  5.3898  3 

0.1293  2.3461  4 

0.1581  0.4617  5 

0.1887  -1.659  6 

0.2260  -4.685  7 

0.2664  -8.13  8 

0.2796  -7.906  9 

0.2938  -7.827  10 

0.3104  -8.062  11 

0.3192  -7.246  12 


Appendix  D 


Response  Column  59 
Actual  by  Predicted  Plot 


1  2  3  4  5  6  7 

Column  59  Predicted  P<.0001  RSq=0.32 
RMSE=1.2645 


Summary  of  Fit 


RSquare  0.319206 

RSquare  Adj  0.25174 

Root  Mean  Square  Error  1 .264453 

Mean  of  Response  3.756098 

Observations  (or  Sum  Wgts)  123 


Analysis  of  Variance 

Source 

DF 

Sum  of  Squares 

Mean  Square 

Model 

11 

83.21163 

7.56469 

Error 

111 

177.47130 

1.59884 

C.  Total 

122 

260.68293 

F  Ratio 
4.7314 
Prob  >  F 
<.0001 


Lack  Of  Fit 


Source  DF 

Lack  Of  Fit  20 

Pure  Error  91 

Total  Error  111 


Sum  of  Squares 
34.55701 
142.91429 
177.47130 


Mean  Square 
1.72785 
1.57049 


F  Ratio 
1.1002 
Prob  >  F 
0.3634 
Max  RSq 
0.4518 


Parameter  Estimates 

Term 

Intercept 

whattypesystemsavionics[1  -0] 
whattypeshrinkinternet[1  -0] 
whattypecomponentclass[1  -0] 
whattypecomponentdevelopment[1-0] 
whattypeenterpriseacctng[1  -0] 
whattypeenterprisepayrolljl  -0] 
whattypeenterpriseOES[1  -0] 
whattypeenterprisescripting[1  -0] 
whattypeenterpriseweb[1-0] 
whatprocessdesig  n  [  1  -0] 
whatproductsnone[1  -0] 


Estimate 
3.6125605 
-1 .532721 
-1.010397 
-1 .930846 
0.927169 
0.8072607 
1.1621648 
-1.754113 
-1.299855 
0.8559698 
0.3173567 
-1.153486 


Std  Error 
0.25271 
0.504171 
0.37601 1 
0.722367 
0.328694 
0.469222 
0.647309 
0.972869 
0.542645 
0.418299 
0.265161 
0.518165 


t  Ratio  Prob>|t| 

14.30  <.0001 

-3.04  0.0030 

-2.69  0.0083 

-2.67  0.0087 

2.82  0.0057 

1.72  0.0881 

1.80  0.0753 

-1.80  0.0741 

-2.40  0.0183 

2.05  0.0431 

1.20  0.2339 

-2.23  0.0280 


Effect  Tests 

Source 

whattypesystemsavionics 

whattypesh  ri  n  ki  nternet 

whattypecomponentclass 

whattypecomponentdevelopment 

whattypeenterpriseacctng 

whattypeenterprisepayroll 

whattypeenterpriseOES 

whattypeenterprisescripting 

whattypeenterpriseweb 

whatprocessdesign 

whatproductsnone 


Nparm  DF 

1  1 

1  1 

1  1 

1  1 

1  1 

1  1 

1  1 

1  1 

1  1 

1  1 

1  1 


Sum  of  Squares 
14.776638 
11.544839 
11.423112 
12.721511 
4.732332 
5.153670 
5.197706 
9.174103 
6.694961 
2.290248 
7.923065 


F  Ratio  Prob  >  F 

9.2421  0.0030 

7.2208  0.0083 

7.1446  0.0087 

7.9567  0.0057 

2.9599  0.0881 

3.2234  0.0753 

3.2509  0.0741 

5.7380  0.0183 

4.1874  0.0431 

1.4324  0.2339 

4.9555  0.0280 


InhouseNonCore  =  3.61  +  sys-avia(-1 .53)  +  shrink-int(-I.OI)  +  comp-class(-1.93)  +  comp-dev(0.93)  +  ent-acct(0.81)  +  ent- 
pay(1.16)  +  ent-OES(-1.75)  +  ent-script(-1 .30)  +  ent-web(0.86)  +  proc-des(0.32)  +  prod-none(-1.15) 
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Stepwise  Fit  -  Combined  Survey  Data  -  Consequences  (InhouseTurnover) 

Response: 

Column  59 

Stepwise  Regression  Control 

Prob  to  Enter  0.250 

Prob  to  Leave  0.250 

Direction: 

Rules: 


Current  Estimates 

SSE  DFE 

74.658386  113 


MSE  RSquare 

0.6606937  0.2671 


RSquare  Adj 
0.2152 


Cp 

-7.47883 


AIC 

-41.914 


Lock 

X 


X  Intercept 

whattypesystemsavionics{0-1} 

X  whattypesystemsembedded{0-1} 

whattypesystemscommunications{0-1} 
X  whattypesystemsdevice{0-1} 

whattypeshrinkbusiness{0-1} 
whattypeshrinkutilities{0-1 } 
whattypeshrinkinternet{0-1 } 
whattypecomponentdomain{0-1} 

X  whattypecomponentCASE{0-1} 

whattypecomponentclass{0- 1 } 

X  whattypecomponentOS{0-1} 

whattypecomponentdevelopment{0-1} 
whattypeenterpriseacctng{0-1 } 
whattypeenterprisemanufact{0-1 } 
whattypeenterprisepayroll{0-1 } 
whattypeenterpriseOES{0- 1 } 
whattypeenterprisescripting{0-1} 
whattypeenterpriseweb{0-1 } 
whatprocessrequirements{0-1} 
whatprocessdesign{0- 1 } 

X  whatprocesstesting{0-1} 

whatprocessmaintenance{0-1} 
whatprocessreengineering{0-1 } 
whatprocessappsuppt{0-1 } 
whatprocesstraining{0-1} 
whatprocessspecification{0-1 } 
whatprocessdocumentation{0-1 } 
whatprocesscoding{0- 1 } 
whatprocessfielding{0-1} 
whatprocessCM{0-1} 
whatprocesstoolsuppt{0-1} 

X  whatprocessSWEngSuppt{0-1} 

X  whatprocessnone{0-1} 

whatproductscustom{0-1 } 
whatproductsCOTS{0-1} 

X  whatproductscommoncust{0-1} 

whatproductsnone{0-1 } 


Step  History 

Step  Parameter 

1  whatprocessnone{0-1} 

2  whatprocessSWEngSuppt{0-1} 

3  whattypesystemsdevice{0-1} 

4  whatprocesstesting{0-1} 

5  whattypecomponentOS{0-1} 

6  whattypesystemsembedded{0-1} 

7  whattypecomponentCASE{0-1} 

8  whatproductscommoncust{0-1} 


Action 

Entered 

Entered 

Entered 

Entered 

Entered 

Entered 

Entered 

Entered 


Estimate  nDF 

2.58039773  1 

0  1 

-0.2095914  1 

0  1 

0.26185638  1 

0  1 

0  1 

0  1 

0  1 

0.2931694  1 

0  1 

0.36309644  1 

0  1 

0  1 

0  1 

0  1 

0  1 

0  1 

0  1 

0  1 

0  1 

0.18154188  1 

0  1 

0  1 

0  1 

0  1 

0  1 

0  1 

0  1 

0  1 

0  1 

0  1 

-0.2941107  1 

1 .05340931  1 

0  1 

0  1 

-0.1229504  1 

0  1 


SS 

0 

0.00014 

2.115388 

0.180836 

2.103914 

0.000093 

0.128705 

0.01089 

0.692642 

1.597295 

0.562394 

3.555997 

0.187154 

0.294737 

0.019694 

0.075741 

0.241936 

0.323474 

0.639809 

0.080945 

0.527686 

3.329363 

0.271989 

0.441306 

0.044103 

0.203274 

0.271118 

0.087154 

0.133614 

0.665279 

0.00029 

0.476032 

7.426122 

8.37996 

0.331963 

0.043984 

1.202459 

0.132374 


"Sig  Prob" 
0.0013 
0.0112 
0.0250 
0.0585 
0.0568 
0.0760 
0.1136 
0.1800 


Seq  SS 
8.402186 
4.936752 
3.704532 
2.567374 
2.543933 
2.161457 
1.691773 
1.202459 


RSquare 

0.0825 

0.1309 

0.1673 

0.1925 

0.2175 

0.2387 

0.2553 

0.2671 


"F  Ratio"  "Prob>F" 

0.000  1.0000 

0.000  0.9885 

3.202  0.0762 

0.272  0.6031 

3.184  0.0770 

0.000  0.9906 

0.193  0.6609 

0.016  0.8985 

1.049  0.3080 

2.418  0.1228 

0.850  0.3585 

5.382  0.0221 

0.281  0.5968 

0.444  0.5066 

0.030  0.8638 

0.114  0.7366 

0.364  0.5474 

0.487  0.4865 

0.968  0.3273 

0.122  0.7280 

0.797  0.3738 

5.039  0.0267 

0.410  0.5235 

0.666  0.4162 

0.066  0.7974 

0.306  0.5814 

0.408  0.5242 

0.131  0.7182 

0.201  0.6549 

1.007  0.3178 

0.000  0.9834 

0.719  0.3984 

11.240  0.0011 

12.684  0.0005 

0.500  0.4809 

0.066  0.7977 

1.820  0.1800 

0.199  0.6564 


Cp  p 

2.8372  2 

-1 .545  3 

-4.335  4 

-5.654  5 

-6.943  6 

-7.737  7 

-7.924  8 

-7.479  9 


Appendix  D 


Response  Column  59 
Actual  by  Predicted  Plot 


Column  59  Predicted  P<.0001  RSq=0.27 
RMSE=0.8128 


Summary  of  Fit 


RSquare  0.267113 

RSquare  Adj  0.215227 

Root  Mean  Square  Error  0.812831 

Mean  of  Response  4.032787 

Observations  (or  Sum  Wgts)  122 


Analysis  of  Variance 


Source  DF 

Model  8 

Error  113 

C.  Total  121 


Sum  of  Squares 
27.21047 
74.65839 
101.86885 


Mean  Square 
3.40131 
0.66069 


F  Ratio 
5.1481 
Prob  >  F 
<.0001 


Lack  Of  Fit 


Source  DF 

Lack  Of  Fit  19 

Pure  Error  94 

Total  Error  113 


Sum  of  Squares 
24.099700 
50.558686 
74.658386 


Mean  Square  F  Ratio 

1 .26841  2.3583 

0.53786  Prob  >  F 

0.0035 
Max  RSq 
0.5037 


Parameter  Estimates 


Term 

Estimate 

Std  Error 

t  Ratio  Prob>|t| 

Intercept 

4.1068186 

0.14003 

29.33  <.0001 

whattypesystemsembeddedfl  -0] 

0.4191828 

0.234266 

1.79  0.0762 

whattypesystemsdevice[1  -0] 

-0.523713 

0.29348 

-1 .78  0.0770 

whattypecomponentCASE[1  -0] 

-0.586339 

0.3771 

-1.55  0.1228 

whattypecomponentOSjl  -0] 

-0.726193 

0.313019 

-2.32  0.0221 

whatprocesstesting[1  -0] 

-0.363084 

0.161743 

-2.24  0.0267 

whatprocessSWEngSuppt[1  -0] 

0.5882214 

0.175453 

3.35  0.0011 

whatprocessnone[1-0] 

-2.106819 

0.59157 

-3.56  0.0005 

whatproductscommoncust[1  -0] 

0.2459008 

0.182274 

1.35  0.1800 

Effect  Tests 

Source 

Nparm  DF 

Sum  of  Squares 

F  Ratio 

Prob  >  F 

whattypesystemsembedded 

1  1 

2.1153881 

3.2018 

0.0762 

whattypesystemsdevice 

1  1 

2.1039141 

3.1844 

0.0770 

whattypecomponentCASE 

1  1 

1 .5972955 

2.4176 

0.1228 

whattypecomponentOS 

1  1 

3.5559969 

5.3822 

0.0221 

whatprocesstesting 

1  1 

3.3293628 

5.0392 

0.0267 

whatprocessSWEngSuppt 

1  1 

7.4261224 

1 1 .2399 

0.0011 

whatprocessnone 

1  1 

8.3799604 

12.6836 

0.0005 

whatproductscommoncust 

1  1 

1 .2024595 

1 .8200 

0.1800 

InhouseTurnover  =  4.11  +  sys-embed(0.42)  +  sys-dev(-0.52)  +  comp-CASE(-0.59)  +  comp-OS(-0.73)  +  proc-test(-0.36)  + 
proc-SWEngSup(0.59)  +  proc-none(-2.11)  +  prod-comcust(0.25) 
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Stepwise  Fit  -  Combined  Survey  Data  -  Consequences  (LearningCurve) 

Response: 

Column  59 


Stepwise  Regression  Control 

Prob  to  Enter  0.250 

Prob  to  Leave  0.250 

Direction: 

Rules: 


Current  Estimates 

SSE  DFE 

144.44252  114 


MSE  RSquare 

1.2670396  0.2920 


RSquare  Adj 
0.2361 


Cp  AIC 

-6.229559  38.92241 


Lock 

X 


X  Intercept 

whattypesystemsavionics{0-1} 
whattypesystemsembedded{0-1 } 
whattypesystemscommunications{0-1} 
whattypesystemsdevice{0-1} 
whattypeshrinkbusiness{0-1} 

X  whattypeshrinkutilities{0-1} 

whattypeshrinkinternet{0-1 } 
whattypecomponentdomain{0-1} 

X  whattypecomponentCASE{0-1} 

X  whattypecomponentclass{0-1} 

whattypecomponentOS{0-1} 
whattypecomponentdevelopment{0-1} 
whattypeenterpriseacctng{0-1 } 
whattypeenterprisemanufact{0-1 } 

X  whattypeenterprisepayroll{0-1} 

whattypeenterpriseOES{0- 1 } 

X  whattypeenterprisescripting{0-1} 

whattypeenterpriseweb{0-1 } 
whatprocessrequirements{0-1} 
whatprocessdesign{0- 1 } 
whatprocesstesting{0- 1 } 
whatprocessmaintenance{0-1} 
whatprocessreengineering{0-1} 
whatprocessappsuppt{0-1 } 

X  whatprocesstraining{0-1} 

whatprocessspecification{0-1 } 
whatprocessdocumentation{0-1 } 
whatprocesscoding{0- 1 } 
whatprocessfielding{0-1} 
whatprocessCM{0-1} 
whatprocesstoolsuppt{0-1} 
whatprocessSWEngSuppt{0- 1 } 

X  whatprocessnone{0-1} 

X  whatproductscustom{0-1} 

whatproductsCOTS{0-1} 

X  whatproductscommoncust{0-1} 

whatproductsnone{0-1 } 


Step  History 

Step  Parameter 

1  whattypeenterprisepayroll{0-1} 

2  whattypecomponentCASE{0-1} 

3  whatprocessnone{0-1} 

4  whatproductscommoncust{0-1} 

5  whatproductscustom{0-1} 

6  whattypecomponentclass{0-1} 

7  whatprocesstraining{0-1} 

8  whattypeenterprisescripting{0-1} 

9  whattypeshrinkutilities{0-1} 


Action 

Entered 

Entered 

Entered 

Entered 

Entered 

Entered 

Entered 

Entered 

Entered 


Estimate 

2.01355221 

0 

0 

0 

0 

0 

-0.3512776 

0 

0 

0.89590772 

0.46863596 

0 

0 

0 

0 

0.6426483 

0 

-0.331167 

0 

0 

0 

0 

0 

0 

0 

0.20237974 

0 

0 

0 

0 

0 

0 

0 

0.9144863 

-0.2248735 

0 

-0.4013195 

0 


"Sig  Prob" 
0.0017 
0.0035 
0.0074 
0.0287 
0.1166 
0.0922 
0.1639 
0.1911 
0.1788 


nDF 

1 

1 

1 

1 

1 

1 

1 

1 

1 

1 

1 

1 

1 

1 

1 

1 

1 

1 

1 

1 

1 

1 

1 

1 

1 

1 

1 

1 

1 

1 

1 

1 

1 

1 

1 

1 

1 

1 

Seq  SS 
15.91402 
12.8615 
10.20633 
6.532937 
3.287834 
3.732736 
2.519874 
2.207473 
2.31897 


SS 

0 

0.617392 

0.284588 

0.059213 

0.047803 

0.761823 

2.31897 

0.777335 

0.525151 

14.84715 

2.815723 

0.351262 

0.116196 

0.280248 

0.155142 

6.569375 

0.947336 

2.40121 

0.218624 

0.000174 

0.589305 

1.434034 

1.251476 

0.130633 

0.027837 

3.024678 

0.085991 

0.276354 

0.079029 

0.118675 

0.842109 

0.078755 

0.415265 

6.219786 

4.776427 

0.030534 

12.00463 

0.001171 


RSquare 

0.0780 

0.1410 

0.1911 

0.2231 

0.2392 

0.2575 

0.2698 

0.2807 

0.2920 


"F  Ratio" 
0.000 
0.485 
0.223 
0.046 
0.037 
0.599 
1.830 
0.611 
0.412 
11.718 
2.222 
0.275 
0.091 
0.220 
0.122 
5.185 
0.746 
1.895 
0.171 
0.000 
0.463 
1.133 
0.988 
0.102 
0.022 
2.387 
0.067 
0.217 
0.062 
0.093 
0.663 
0.062 
0.326 
4.909 
3.770 
0.024 
9.475 
0.001 


Cp 

7.3283 

0.6226 

-4.286 

-6.708 

-6.933 

-7.46 

-7.166 

-6.66 

-6.23 


"Prob>F" 

1.0000 

0.4876 

0.6376 

0.8299 

0.8470 

0.4405 

0.1788 

0.4359 

0.5221 

0.0009 

0.1388 

0.6007 

0.7635 

0.6402 

0.7281 

0.0247 

0.3896 

0.1713 

0.6798 

0.9907 

0.4977 

0.2894 

0.3225 

0.7497 

0.8829 

0.1251 

0.7958 

0.6425 

0.8040 

0.7611 

0.4173 

0.8044 

0.5693 

0.0287 

0.0547 

0.8774 

0.0026 

0.9759 

P 

2 

3 

4 

5 

6 

7 

8 

9 

10 
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Response  Column  59 
Actual  by  Predicted  Plot 


7-  ■  ■ 


1-  ■  ■ 


1  1  I  1  I  1  I  1  I  1  I  1 

1  2  3  4  5  6  7 

Column  59  Predicted  P<.0001  RSq=0.29 
RMSE=1 .1256 


Summary  of  Fit 


RSquare  0.292032 

RSquare  Adj  0.23614 

Root  Mean  Square  Error  1.125629 

Mean  of  Response  4.08871 

Observations  (or  Sum  Wgts)  124 


Analysis  of  Variance 


Source  DF 

Model  9 

Error  114 

C.  Total  123 


Sum  of  Squares 
59.58167 
144.44252 
204.02419 


Mean  Square  F  Ratio 

6.62019  5.2249 

1 .26704  Prob  >  F 

<.0001 


Lack  Of  Fit 


Source  DF 

Lack  Of  Fit  17 

Pure  Error  97 

Total  Error  114 


Sum  of  Squares 
30.53235 
113.91017 
144.44252 


Mean  Square  F  Ratio 

1.79602  1.5294 

1.17433  Prob  >  F 

0.1004 
Max  RSq 
0.4417 


Parameter  Estimates 

Term 

Intercept 

whattypeshrinkutilities[1  -0] 
whattypecomponentCASE[1  -0] 
whattypecomponentclass[1  -0] 
whattypeenterprisepayroll[1  -0] 
whattypeenterprisescripting[1-0] 
whatprocesstraining[1  -0] 
whatprocessnone[1  -0] 
whatproductscustom[1  -0] 
whatproductscommoncust[1  -0] 


Estimate 

3.8289726 

0.7025552 

-1.791815 

-0.937272 

-1.285297 

0.662334 

-0.404759 

-1.828973 

0.4497471 

0.802639 


Std  Error 
0.218912 
0.519312 
0.52344 
0.628732 
0.564464 
0.481124 
0.261971 
0.825495 
0.231639 
0.26076 


t  Ratio 
17.49 
1.35 
-3.42 
-1.49 
-2.28 
1.38 
-1.55 
-2.22 
1.94 
3.08 


Prob>|t| 

<.0001 

0.1788 

0.0009 

0.1388 

0.0247 

0.1713 

0.1251 

0.0287 

0.0547 

0.0026 


Effect  Tests 

Source 

whattypeshrinkutilities 

whattypecomponentCASE 

whattypecomponentclass 

whattypeenterprisepayroll 

whattypeenterprisescripting 

whatprocesstraining 

whatprocessnone 

whatproductscustom 

whatproductscommoncust 


Nparm  DF 

1  1 

1  1 

1  1 

1  1 

1  1 

1  1 

1  1 

1  1 

1  1 


Sum  of  Squares  F  Ratio  Prob  >  F 

2.318970  1.8302  0.1788 

14.847153  11.7180  0.0009 

2.815723  2.2223  0.1388 

6.569375  5.1848  0.0247 

2.401210  1.8951  0.1713 

3.024678  2.3872  0.1251 

6.219786  4.9089  0.0287 

4.776427  3.7698  0.0547 

12.004625  9.4745  0.0026 


LearningCurve  =  3.83  +  shrink-util(0.70)  +  comp-CASE(-1.79)  +  comp-class(-0.94)  +  ent-pay(-1 .29)  +  ent-script(0.66)  + 
proc-train(-0.40)  +  proc-none(-1 .83)  +  prod-cust(0.45)  +  prod-comcust(0.80) 
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Stepwise  Fit  -  Combined  Survey  Data  -  Consequences  (Risk) 

Response: 

Column  59 


Stepwise  Regression  Control 

Prob  to  Enter  0.250 

Prob  to  Leave  0.250 

Direction: 

Rules: 


Current  Estimates 

SSE  DFE 

210.83322  110 


MSE  RSquare 

1.9166656  0.2994 


RSquare  Adj 
0.2102 


Cp  AIC 

0.8635043  95.34421 


Entered  Parameter 

X  Intercept 

X  whattypesystemsavionics{0-1} 

whattypesystemsembedded{0-1 } 

X  whattypesystemscommunications{0-1} 

whattypesystemsdevice{0-1} 
whattypeshrinkbusiness{0-1} 

X  whattypeshrinkutilities{0-1} 

whattypeshrinkinternet{0-1 } 
whattypecomponentdomain{0-1} 
whattypecomponentCASE{0-1 } 
whattypecomponentclass{0- 1 } 

X  whattypecomponentOS{0-1} 

X  whattypecomponentdevelopment{0-1} 

whattypeenterpriseacctng{0-1} 
whattypeenterprisemanufact{0-1 } 
whattypeenterprisepayroll{0-1} 

X  whattypeenterpriseOES{0-1} 

X  whattypeenterprisescripting{0-1} 

X  whattypeenterpriseweb{0-1} 

whatprocessrequirements{0-1} 
whatprocessdesign{0- 1 } 
whatprocesstesting{0- 1 } 
whatprocessmaintenance{0-1} 

X  whatprocessreengineering{0-1} 

whatprocessappsuppt{0-1 } 

X  whatprocesstraining{0-1} 

whatprocessspecification{0-1 } 

X  whatprocessdocumentation{0-1} 

whatprocesscoding{0- 1 } 
whatprocessfielding{0-1} 

X  whatprocessCM{0-1} 

whatprocesstoolsuppt{0-1} 

X  whatprocessSWEngSuppt{0-1} 

whatprocessnone{0-1 } 
whatproductscustom{0-1 } 
whatproductsCOTS{0-1} 
whatproductscommoncust{0-1} 
whatproductsnone{0-1 } 


Step  History 

Step  Parameter 


9 

10 

11 

12 

13 

14 


whattypeshrinkutilities{0-1} 
whatproductsnone{0-1} 
whattypecomponentOS{0-1 } 
whattypeenterprisescripting{0-1} 
whattypesystemscommunications{0-1} 
whattypeenterpriseOES{0-1} 
whatprocessCM{0-1} 
whatprocessdocumentation{0-1} 
whatprocessSWEngSuppt{0-1 } 
whatprocesstraining{0-1 } 
whattypesystemsavionics{0-1 } 
whattypeenterpriseweb{0-1 } 
whattypecomponentdevelopment{0-1} 
whatprocessreengineering{0-1 } 


Estimate 

0.87331128 

0.4455735 


-0.8343679 

0 

0 

0 

0 

0.5994816 

-0.2998499 


1.23929262 

0.86899438 

0.49917581 


0 

0 

-0.2168184 

0 

0.32223359 

0 

-0.4366682 


SS 

0 

4.622714 

I. 109257 

I I . 85474 
2.018904 
0.062988 
12.01854 

0.00562 

1.869075 

I. 196465 
0.008436 
10.08205 
5.700889 
0.544451 

0.33328 

2.509847 

II. 14129 
16.41125 
9.228481 
0.285006 
0.003152 
0.527154 
1.807967 
2.829951 
0.089672 
6.657614 
0.026817 
8.244435 
0.107028 
0.002879 

4.71844 

0.021683 

3.108055 

0.058609 

0.080829 

1.105936 

0.786251 

18.89502 


"F  Ratio"  "Prob>F" 

0.000  1.0000 

2.412  0.1233 

0.577  0.4493 

6.185  0.0144 

1.054  0.3069 

0.033  0.8571 

6.271  0.0137 

0.003  0.9571 

0.975  0.3256 

0.622  0.4320 

0.004  0.9475 

5.260  0.0237 

2.974  0.0874 

0.282  0.5963 

0.173  0.6786 

1.313  0.2543 

5.813  0.0176 

8.562  0.0042 

4.815  0.0303 

0.148  0.7016 

0.002  0.9679 

0.273  0.6022 

0.943  0.3337 

1.476  0.2269 

0.046  0.8299 

3.474  0.0650 

0.014  0.9065 

4.301  0.0404 

0.055  0.8144 

0.001  0.9693 

2.462  0.1195 

0.011  0.9159 

1.622  0.2056 

0.030  0.8621 

0.042  0.8384 

0.575  0.4500 

0.408  0.5243 

9.858  0.0022 


Action 

Entered 

Entered 

Entered 

Entered 

Entered 

Entered 

Entered 

Entered 

Entered 

Entered 

Entered 

Entered 

Entered 

Entered 


"Sig  Prob"  Seq  SS 

0.0203  12.93633 

0.0493  9.01417 

0.0459  9.072735 

0.0312  10.28701 

0.0585  7.723081 

0.0749  6.708421 

0.1256  4.889272 

0.0869  6.02228 

0.2150  3.124689 

0.1974  3.358734 

0.1881  3.48031 

0.1608  3.921542 

0.0642  6.726251 

0.2269  2.829951 


RSquare  Cp  p 

0.0430  9.9466  2 

0.0729  7.8479  3 

0.1031  5.7227  4 

0.1373  3.0453  5 

0.1629  1.5337  6 

0.1852  0.4834  7 

0.2015  0.2603  8 

0.2215  -0.478  9 

0.2319  0.1013  10 

0.2430  0.5741  11 

0.2546  0.9917  12 

0.2676  1.2086  13 

0.2900  0.1503  14 

0.2994  0.8635  15 
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Response  Column  59 
Actual  by  Predicted  Plot 


i  i  i  i  i  i  i  i  i  i  i 

1  2  3  4  5  6  7 

Column  59  Predicted  P=0.0002  RSq=0.30 
RMSE=1 .3844 


Summary  of  Fit 


RSquare 

0.29939 

RSquare  Adj 

0.210221 

Root  Mean  Square  Error 

1.384437 

Mean  of  Response 

4.024 

Observations  (or  Sum  Wgts) 

125 

Analysis  of  Variance 

Source  DF  Sum  of  Squares  Mean  Square 

Model  14 

90.09478 

6.43534 

Error  110 

210.83322 

1.91667 

C.  Total  124 

300.92800 

Lack  Of  Fit 

Source  DF 

Sum  of  Squares 

Mean  Square 

Lack  Of  Fit  51 

121.46941 

2.38175 

Pure  Error  59 

89.36381 

1.51464 

Total  Error  110 

210.83322 

Parameter  Estimates 

Term 

Estimate 

Intercept 

4.4175711 

whattypesystemsavionics[1  -0] 

-0.891147 

whattypesystemscommunications[1-0] 

-0.799222 

whattypeshrinkutilities[1  -0] 

1.6687358 

whattypecomponentOS[1  -0] 

-1.198963 

whattypecomponentdevelopment[1-0] 

0.5996998 

whattypeenterpriseOES[1  -0] 

-2.478585 

whattypeenterprisescripting[1  -0] 

-1.737989 

whattypeenterpriseweb[1  -0] 

-0.998352 

whatprocessreengineering[1  -0] 

0.4336368 

whatprocesstraining[1-0] 

-0.644467 

whatprocessdocumentationjl  -0] 

0.8733364 

whatprocessCMjl  -0] 

-0.53795 

whatprocessSWEngSuppt[1  -0] 

0.3977722 

whatproductsnone[1-0] 

-1 .775026 

Effect  Tests 

Source 

Nparm 

DF 

whattypesystemsavionics 

1 

1 

whattypesystemscommunications 

1 

1 

whattypeshrinkutilities 

1 

1 

whattypecomponentOS 

1 

1 

whattypecomponentdevelopment 

1 

1 

whattypeenterpriseOES 

1 

1 

whattypeenterprisescripting 

1 

1 

whattypeenterpriseweb 

1 

1 

whatprocessreengineering 

1 

1 

whatprocesstraining 

1 

1 

whatprocessdocumentation 

1 

1 

whatprocessCM 

1 

1 

whatprocessSWEngSuppt 

1 

1 

whatprod  uctsnone 

1 

1 

F  Ratio 
3.3576 
Prob  >  F 
0.0002 


F  Ratio 
1.5725 
Prob  >  F 
0.0470 
Max  RSq 
0.7030 


Std  Error 
0.201521 
0.573818 
0.321362 
0.6664 
0.522763 
0.347725 
1 .028038 
0.593949 
0.454979 
0.35687 
0.345792 
0.421089 
0.342859 
0.312366 
0.565333 


t  Ratio 
21.92 
-1.55 
-2.49 
2.50 
-2.29 
1.72 
-2.41 
-2.93 
-2.19 
1.22 
-1.86 
2.07 
-1.57 
1.27 
-3.14 


Prob>|t| 

<.0001 

0.1233 

0.0144 

0.0137 

0.0237 

0.0874 

0.0176 

0.0042 

0.0303 

0.2269 

0.0650 

0.0404 

0.1195 

0.2056 

0.0022 


Sum  of  Squares 
4.622714 
1 1 .854745 
12.018542 
10.082045 
5.700889 
11.141294 
16.411254 
9.228481 
2.829951 
6.657614 
8.244435 
4.718440 
3.108055 
18.895017 


F  Ratio  Prob  >  F 

2.4119  0.1233 

6.1851  0.0144 

6.2705  0.0137 

5.2602  0.0237 

2.9744  0.0874 

5.8129  0.0176 

8.5624  0.0042 

4.8149  0.0303 

1.4765  0.2269 

3.4735  0.0650 

4.3014  0.0404 

2.4618  0.1195 

1.6216  0.2056 

9.8583  0.0022 


Risk  =  4.42  +  sys-avia(-0.89)  +  sys-comm(-0.80)  +  shrinkutil(1 .67)  +  comp-OS(-1.20)  +  comp-dev(0.60)  +  ent-OES(-2.48) 
+  ent-script(-1 .74)  +  ent-web(-I.OO)  +  proc-reeng(0.43)  +  proc-train(-0.64)  +  proc-doc(0.87)  +  proc-CM(-0.54)  +  proc- 
SWEngSup(0.40)  +  prod-none(-1 .78) 
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Stepwise  Fit  -  Combined  Survey  Data  -  Consequences  (Quality) 

Response: 

Column  59 


Stepwise  Regression  Control 

Prob  to  Enter  0.250 

Prob  to  Leave  0.250 

Direction: 

Rules: 


Current  Estimates 

SSE  DFE 

189.43786  113 


MSE  RSquare 

1.6764412  0.2326 


RSquare  Adj  Cp  AIC 

0.1647  -7.669622  74.54865 


Entered  Parameter 

X  Intercept 

whattypesystemsavionics{0-1} 

X  whattypesystemsembedded{0-1} 

whattypesystemscommunications{0-1} 
whattypesystemsdevice{0- 1 } 
whattypeshrinkbusiness{0-1} 

X  whattypeshrinkutilities{0-1} 

X  whattypeshrinkinternet{0-1} 

whattypecomponentdomain{0-1} 
whattypecomponentCASE{0-1 } 
whattypecomponentclass{0- 1 } 

X  whattypecomponentOS{0-1} 

whattypecomponentdevelopment{0-1} 
whattypeenterpriseacctng{0-1} 
whattypeenterprisemanufact{0-1 } 
whattypeenterprisepayroll{0-1} 

X  whattypeenterpriseOES{0-1} 

whattypeenterprisescripting{0-1} 

whattypeenterpriseweb{0-1 } 

whatprocessrequirements{0-1} 

whatprocessdesign{0-1} 

whatprocesstesting{0-1} 

whatprocessmaintenance{0-1} 

X  whatprocessreengineering{0-1} 

whatprocessappsuppt{0-1 } 
whatprocesstraining{0-1} 
whatprocessspecification{0-1 } 
whatprocessdocumentation{0-1 } 
whatprocesscoding{0- 1 } 

X  whatprocessfielding{0-1} 

whatprocessCM{0-1} 
whatprocesstoolsuppt{0-1} 
whatprocessSWEngSuppt{0- 1 } 
whatprocessnone{0-1 } 
whatproductscustom{0-1 } 

X  whatproductsCOTS{0-1} 

X  whatproductscommoncust{0-1} 

X  whatproductsnone{0-1} 


Step  History 

Step  Parameter 


9 

10 


whatproductsCOTS{0-1} 
whattypecomponentOS{0-1 } 
whatproductscommoncust{0-1} 
whattypeshrinkinternet{0-1} 
whatprocessreengineering{0-1 } 
whatproductsnone{0-1 } 
whatprocessfielding{0-1} 
whattypesystemsembedded{0-1} 
whattypeshrinkutilities{0-1} 
whattypeenterpriseOES{0- 1 } 


Action 

Entered 

Entered 

Entered 

Entered 

Entered 

Entered 

Entered 

Entered 

Entered 

Entered 


Estimate  nDF 

5.94560058  1 

0  1 

0.26314942  1 

0  1 

0  1 

0  1 

-0.3755373  1 

-0.3354296  1 

0  1 

0  1 

0  1 

-0.5995587  1 

0  1 

0  1 

0  1 

0  1 

-0.566412  1 

0  1 

0  1 

0  1 

0  1 

0  1 

0  1 

-0.3090394  1 

0  1 

0  1 

0  1 

0  1 

0  1 

0.29849357  1 

0  1 

0  1 

0  1 

0  1 

0  1 

0.44743634  1 

0.22413105  1 

-0.4120728  1 


"Sig  Prob" 
0.0171 
0.0161 
0.0353 
0.0664 
0.0771 
0.1542 
0.1357 
0.1840 
0.2233 
0.2316 


Seq  SS 
11.27763 
11.04828 
8.175523 
6.064783 
5.518937 
3.537692 
3.842127 
3.020196 
2.523122 
2.424813 


SS 

0 

0.702784 
3.272454 
1.167217 
0.424218 
0.62277 
2.466045 
5.360056 
1.764469 
1.137608 
0.019883 
9.896998 
0.327059 
1 .378625 
0.194249 
0.24247 
2.424813 
0.226878 
1.396231 
0.003828 
0.224508 
0.000249 
0.937928 
6.91938 
0.315444 
0.105099 
0.000012 
0.160431 
0.205294 
4.552133 
0.04929 
0.012637 
0.74469 
0.025794 
0.137466 
13.31403 
3.859695 
4.314293 


RSquare 

0.0457 

0.0904 

0.1236 

0.1481 

0.1705 

0.1848 

0.2004 

0.2126 

0.2228 

0.2326 


"F  Ratio"  "Prob>F" 

0.000  1.0000 

0.417  0.5197 

1.952  0.1651 

0.694  0.4065 

0.251  0.6171 

0.369  0.5446 

1.471  0.2277 

3.197  0.0764 

1.053  0.3070 

0.677  0.4125 

0.012  0.9139 

5.904  0.0167 

0.194  0.6607 

0.821  0.3668 

0.115  0.7352 

0.144  0.7055 

1.446  0.2316 

0.134  0.7147 

0.832  0.3638 

0.002  0.9621 

0.133  0.7161 

0.000  0.9903 

0.557  0.4569 

4.127  0.0445 

0.187  0.6664 

0.062  0.8036 

0.000  0.9979 

0.095  0.7586 

0.122  0.7281 

2.715  0.1022 

0.029  0.8647 

0.007  0.9313 

0.442  0.5075 

0.015  0.9019 

0.081  0.7760 

7.942  0.0057 

2.302  0.1320 

2.573  0.1115 


Cp  p 

-2.687  2 

-6.188  3 

-8.259  4 

-9.279  5 

-10.03  6 

-9.789  7 

-9.702  8 

-9.206  9 

-8.462  10 

-7.67  11 


Appendix  D 


Response  Column  59 
Actual  by  Predicted  Plot 


1  1  i  1  i  1  i  1  i  1  i  1 

1  2  3  4  5  6  7 

Column  59  Predicted  P=0.0006  RSq=0.23 
RMSE=1 .2948 


Summary  of  Fit 


RSquare  0.232644 

RSquare  Adj  0.164737 

Root  Mean  Square  Error  1.294775 

Mean  of  Response  4.532258 

Observations  (or  Sum  Wgts)  124 


Analysis  of  Variance 


Source  DF 

Model  10 

Error  113 

C.  Total  123 


Sum  of  Squares 
57.4331 1 
189.43786 
246.87097 


Mean  Square 
5.74331 
1 .67644 


F  Ratio 
3.4259 
Prob  >  F 
0.0006 


Lack  Of  Fit 


Source  DF 

Lack  Of  Fit  29 

Pure  Error  84 

Total  Error  113 


Sum  of  Squares 
58.10016 
131.33770 
189.43786 


Mean  Square  F  Ratio 

2.00345  1.2814 

1.56354  Prob  >  F 

0.1907 
Max  RSq 
0.4680 


Parameter  Estimates 


Term 

Estimate 

Std  Error 

t  Ratio  Prob>|t| 

Intercept 

4.5807612 

0.17842 

25.67  <.0001 

whattypesystemsembeddedjl  -0] 

-0.526299 

0.376695 

-1.40  0.1651 

whattypeshrinkutilities[1  -0] 

0.7510746 

0.619265 

1.21  0.2277 

whattypeshrinkinternet[1  -0] 

0.6708591 

0.375181 

1.79  0.0764 

whattypecomponentOSjl  -0] 

1.1991173 

0.493519 

2.43  0.0167 

whattypeenterpriseOES[1  -0] 

1.1328241 

0.941928 

1.20  0.2316 

whatprocessreengineeringjl  -0] 

0.6180787 

0.304232 

2.03  0.0445 

whatprocessfieldingjl  -0] 

-0.596987 

0.362286 

-1.65  0.1022 

whatproductsCOTSjl  -0] 

-0.894873 

0.317542 

-2.82  0.0057 

whatproductscommoncust[1  -0] 

-0.448262 

0.295427 

-1.52  0.1320 

whatproductsnone[1  -0] 

0.8241457 

0.51374 

1.60  0.1115 

Effect  Tests 

Source 

Nparm  DF 

Sum  of  Squares 

F  Ratio 

Prob  >  F 

whattypesystemsembedded 

1  1 

3.272454 

1 .9520 

0.1651 

whattypeshrinkutilities 

1  1 

2.466045 

1.4710 

0.2277 

whattypeshrinkinternet 

1  1 

5.360056 

3.1973 

0.0764 

whattypecomponentOS 

1  1 

9.896998 

5.9036 

0.0167 

whattypeenterpriseOES 

1  1 

2.424813 

1.4464 

0.2316 

whatprocessreengineering 

1  1 

6.919380 

4.1274 

0.0445 

whatprocessfielding 

1  1 

4.552133 

2.7154 

0.1022 

whatproductsCOTS 

1  1 

13.314025 

7.9418 

0.0057 

whatproductscommoncust 

1  1 

3.859695 

2.3023 

0.1320 

whatproductsnone 

1  1 

4.314293 

2.5735 

0.1115 

Quality  =  4.58  +  sys-embed(-0.53)  +  shrink-util(0.75)  +  shrink-int(0.67)  +  comp-OS(1.20)  +  ent-OES(1.13)  +  proc- 
reeng(0.62)  +proc-field(-0.60)  +  prod-COTS(-0.89)  +  prod-comcust(-0.45)  +  prod-none(0.82) 
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Stepwise  Fit  -  Combined  Survey  Data  -  Consequences  (Rework) 

Response: 

Column  59 


Stepwise  Regression  Control 

Prob  to  Enter  0.250 

Prob  to  Leave  0.250 

Direction: 

Rules: 


Current  Estimates 

SSE  DFE 

173.48063  111 


MSE  RSquare 

1.5628886  0.2137 


RSquare  Adj  Cp  AIC 

0.1428  -4.186418  64.94948 


Lock 

X 


X  Intercept 

whattypesystemsavionics{0-1} 
whattypesystemsembedded{0-1 } 

X  whattypesystemscommunications{0-1} 

whattypesystemsdevice{0-1} 
whattypeshrinkbusiness{0-1} 

X  whattypeshrinkutilities{0-1} 

whattypeshrinkinternet{0-1 } 
whattypecomponentdomain{0-1} 
whattypecomponentCASE{0-1 } 
whattypecomponentclass{0- 1 } 

X  whattypecomponentOS{0-1} 

X  whattypecomponentdevelopment{0-1} 

whattypeenterpriseacctng{0-1 } 
whattypeenterprisemanufact{0-1 } 
whattypeenterprisepayroll{0-1 } 

X  whattypeenterpriseOES{0-1} 

whattypeenterprisescripting{0-1} 
whattypeenterpriseweb{0-1 } 
whatprocessrequirements{0-1} 

X  whatprocessdesign{0-1} 

whatprocesstesting{0-1} 
whatprocessmaintenance{0-1} 

X  whatprocessreengineering{0-1} 

whatprocessappsuppt{0-1 } 
whatprocesstraining{0-1} 
whatprocessspecification{0-1 } 
whatprocessdocumentation{0-1 } 

X  whatprocesscoding{0-1} 

whatprocessfielding{0-1} 
whatprocessCM{0-1} 
whatprocesstoolsuppt{0-1} 
whatprocessSWEngSuppt{0- 1 } 

X  whatprocessnone{0-1} 

whatproductscustom{0-1 } 

X  whatproductsCOTS{0-1} 

whatproductscommoncust{0-1} 
whatproductsnone{0-1 } 


Step  History 

Step  Parameter 

1  whatproductsCOTS{0-1} 

2  whattypeshrinkutilities{0-1} 

3  whattypecomponentdevelopment{0- 1 } 

4  whattypesystemscommunications{0-1 } 

5  whatprocessnone{0-1} 

6  whatprocessreengineering{0-1} 

7  whatprocessdesign{0-1} 

8  whatprocesscoding{0-1} 

9  whattypecomponentOS{0-1} 

10  whattypeenterpriseOES{0-1} 


Estimate  nDF 

4.81287671  1 

0  1 

0  1 

0.30431091  1 

0  1 

0  1 

-0.6082568  1 

0  1 

0  1 

0  1 

0  1 

0.3696466  1 

-0.421562  1 

0  1 

0  1 

0  1 

0.58363878  1 

0  1 

0  1 

0  1 

-0.2590507  1 

0  1 

0  1 

0.2077958  1 

0  1 

0  1 

0  1 

0  1 

0.26027606  1 

0  1 

0  1 

0  1 

0  1 

-0.6424936  1 

0  1 

-0.392169  1 

0  1 

0  1 


SS 

0 

0.854713 

0.2848 

6.868883 

0.420943 

0.265706 

6.340475 

0.22584 

I. 504342 
0.141421 
0.830824 

3.94366 

II. 94422 
0.090616 
0.043839 
0.492515 
2.587659 
0.659303 
0.012623 
0.111326 
5.487904 
0.005561 
0.010112 
3.053797 
0.863636 
0.093804 
0.932767 
0.106412 
3.654039 
0.794367 
1 .604069 
1 .058588 
0.581387 
2.936087 

1.12927 

10.80005 

0.212798 

0.286422 


"F  Ratio"  "Prob>F" 

0.000  1.0000 

0.545  0.4621 

0.181  0.6714 

4.395  0.0383 

0.268  0.6060 

0.169  0.6820 

4.057  0.0464 

0.143  0.7057 

0.962  0.3288 

0.090  0.7651 

0.529  0.4684 

2.523  0.1150 

7.642  0.0067 

0.057  0.8110 

0.028  0.8679 

0.313  0.5769 

1.656  0.2009 

0.420  0.5185 

0.008  0.9289 

0.071  0.7909 

3.511  0.0636 

0.004  0.9528 

0.006  0.9363 

1.954  0.1649 

0.550  0.4598 

0.060  0.8077 

0.595  0.4423 

0.068  0.7955 

2.338  0.1291 

0.506  0.4784 

1.027  0.3132 

0.675  0.4130 

0.370  0.5443 

1.879  0.1733 

0.721  0.3977 

6.910  0.0098 

0.135  0.7139 

0.182  0.6706 


Action 

Entered 

Entered 

Entered 

Entered 

Entered 

Entered 

Entered 

Entered 

Entered 

Entered 


"Sig  Prob"  Seq  SS 

0.0168  10.31003 

0.0614  6.118156 

0.1027  4.575584 

0.0751  5.353451 

0.1113  4.217847 

0.1317  3.734342 

0.1462  3.434737 

0.1694  3.044465 

0.1245  3.766049 

0.2009  2.587659 


RSquare  Cp  p 

0.0467  -1 .844  2 

0.0745  -3.223  3 

0.0952  -3.75  4 

0.1195  -4.707  5 

0.1386  -5.036  6 

0.1555  -5.099  7 

0.1711  -4.996  8 

0.1849  -4.677  9 

0.2019  -4.757  10 

0.2137  -4.186  11 


Appendix  D 


Response  Column  59 
Actual  by  Predicted  Plot 


1  2  3  4  5  6  7 

Column  59  Predicted  P=0.0021  RSq=0.21 
RMSE=1.2502 


Summary  of  Fit 


RSquare  0.213678 

RSquare  Adj  0.142838 

Root  Mean  Square  Error  1.250155 

Mean  of  Response  4.245902 

Observations  (or  Sum  Wgts)  122 


Analysis  of  Variance 

Source 

DF 

Sum  of  Squares 

Mean  Square 

Model 

10 

47.14232 

4.71423 

Error 

111 

173.48063 

1.56289 

C.  Total 

121 

220.62295 

F  Ratio 
3.0164 
Prob  >  F 
0.0021 


Lack  Of  Fit 


Source  DF 

Lack  Of  Fit  33 

Pure  Error  78 

Total  Error  111 


Sum  of  Squares  Mean  Square 

43.42869  1.31602 

130.05195  1.66733 

173.48063 


F  Ratio 
0.7893 
Prob  >  F 
0.7730 
Max  RSq 
0.4105 


Parameter  Estimates 

Term 

Intercept 

whattypesystemscommunications[1-0] 
whattypeshrinkutilities[1  -0] 
whattypecomponentOSjl  -0] 
whattypecomponentdevelopment[1-0] 
whattypeenterpriseOES[1  -0] 
whatprocessdesign[1-0] 
whatprocessreengineering[1-0] 
whatprocesscoding[1-0] 
whatprocessnone[1-0] 
whatproductsCOTSjl  -0] 


Estimate 

4.2150128 

-0.608622 

1.2165136 

-0.739293 

0.843124 

-1.167278 

0.5181014 

-0.415592 

-0.520552 

1.2849872 

0.7843379 


Std  Error 
0.312235 
0.290314 
0.603977 
0.465405 
0.304984 
0.907162 
0.276487 
0.29731 1 
0.340441 
0.937515 
0.298369 


t  Ratio 
13.50 
-2.10 
2.01 
-1.59 
2.76 
-1.29 
1.87 
-1.40 
-1.53 
1.37 
2.63 


Prob>|t| 

<.0001 

0.0383 

0.0464 

0.1150 

0.0067 

0.2009 

0.0636 

0.1649 

0.1291 

0.1733 

0.0098 


Effect  Tests 

Source 

whattypesystemscommunications 

whattypeshrinkutilities 

whattypecomponentOS 

whattypecomponentdevelopment 

whattypeenterpriseOES 

whatprocessdesign 

whatprocessreengineering 

whatprocesscoding 

whatprocessnone 

whatproductsCOTS 


Nparm  DF 

1  1 

1  1 

1  1 

1  1 

1  1 

1  1 

1  1 

1  1 

1  1 

1  1 


Sum  of  Squares 
6.868883 
6.340475 
3.943660 
11.944224 
2.587659 
5.487904 
3.053797 
3.654039 
2.936087 
10.800048 


F  Ratio  Prob  >  F 

4.3950  0.0383 

4.0569  0.0464 

2.5233  0.1150 

7.6424  0.0067 

1 .6557  0.2009 

3.5114  0.0636 

1.9539  0.1649 

2.3380  0.1291 

1.8786  0.1733 

6.9103  0.0098 


Rework  =  4.22  +  sys-comm(-0.61)  +  shrink-util(1 .22)  +  comp-OS(-0.74)  +  comp-dev(0.84)  +  ent-OES(-1.17)  +  proc- 
des(0.52)  +proc-reeng(-0.42)  +  proc-coding(-0.52)  +  proc-none(1.28)  +  prod-COTS(0.78) 
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Stepwise  Fit  -  Combined  Survey  Data  -  Consequences  (Visibility) 

Response: 

Column  59 

Stepwise  Regression  Control 

Prob  to  Enter  0.250 

Prob  to  Leave  0.250 

Direction: 

Rules: 


Current  Estimates 

SSE  DFE 

158.35814  111 


MSE  RSquare 

1.4266499  0.3228 


RSquare  Adj 
0.2557 


Cp 

-0.492922 


AIC 

55.079 


Entered  Parameter 

X  Intercept 

whattypesystemsavionics{0-1} 
whattypesystemsembedded{0-1 } 

X  whattypesystemscommunications{0-1 } 

whattypesystemsdevice{0-1} 

X  whattypeshrinkbusiness{0-1} 

whattypeshrinkutilities{0-1 } 
whattypeshrinkinternet{0-1 } 
whattypecomponentdomain{0-1} 
whattypecomponentCASE{0-1 } 
whattypecomponentclass{0- 1 } 
whattypecomponentOS{0-1 } 
whattypecomponentdevelopment{0-1} 
whattypeenterpriseacctng{0-1} 
whattypeenterprisemanufact{0-1 } 
whattypeenterprisepayroll{0-1} 

X  whattypeenterpriseOES{0-1} 

whattypeenterprisescripting{0-1} 
whattypeenterpriseweb{0-1 } 
whatprocessrequirements{0-1} 
whatprocessdesign{0- 1 } 
whatprocesstesting{0-1} 
whatprocessmaintenance{0-1} 

X  whatprocessreengineering{0-1} 

X  whatprocessappsuppt{0-1} 

whatprocesstraining{0-1} 
whatprocessspecification{0-1 } 
whatprocessdocumentation{0-1 } 

X  whatprocesscoding{0-1} 

X  whatprocessfielding{0-1} 

whatprocessCM{0-1} 
whatprocesstoolsuppt{0-1} 
whatprocessSWEngSuppt{0- 1 } 

X  whatprocessnone{0-1} 

X  whatproductscustom{0-1} 

whatproductsCOTS{0-1} 

X  whatproductscommoncust{0-1} 

X  whatproductsnone{0-1} 


Step  History 

Step  Parameter 


9 

10 

11 

12 

13 


whatproductsnone{0-1 } 
whatproductscustom{0-1 } 
whattypecomponentclass{0-1 } 
whatprocessfielding{0-1} 
whatprocessappsuppt{0-1 } 
whattypesystemscommunications{0-1} 
whatprocessnone{0-1 } 
whattypeshrinkbusiness{0-1 } 
whattypeenterpriseOES{0- 1 } 
whattypecomponentclass{0-1 } 
whatprocesscoding{0-1} 
whatprocessreengineering{0-1 } 
whatproductscommoncust{0-1} 


Estimate  nDF 

4.86890586  1 

0  1 

0  1 

-0.2541736  1 

0  1 

-0.2772284  1 

0  1 

0  1 

0  1 

0  1 

0  1 

0  1 

0  1 

0  1 

0  1 

0  1 

-1.0290139  1 

0  1 

0  1 

0  1 

0  1 

0  1 

0  1 

-0.1906506  1 

-0.3483678  1 

0  1 

0  1 

0  1 

0.18443306  1 

0.65864736  1 

0  1 

0  1 

0  1 

1 .09808057  1 

-0.3885451  1 

0  1 

-0.18332  1 

-1.2801653  1 


SS 

0 

0.091465 
1 .64202 
5.048573 
0.392463 
6.279905 
0.359519 
0.401354 
1.697239 
0.158115 
1.331849 
0.103933 
0.073543 
0.112491 
0.156561 
0.077588 
7.93466 
1.82071 
1.584916 
0.193714 
0.10288 
0.051857 
0.563034 
2.50037 
7.903873 
0.015385 
0.171667 
0.199242 
2.203369 
18.51537 
0.000158 
0.663244 
1.494537 
6.561818 
12.72592 
1.169263 
2.347732 
26.43738 


"F  Ratio" 
0.000 
0.064 
1.153 

3.539 
0.273 
4.402 
0.250 
0.279 
1.192 
0.110 
0.933 
0.072 
0.051 
0.078 
0.109 
0.054 
5.562 
1.279 
1.112 
0.135 
0.072 
0.036 
0.392 
1.753 

5.540 
0.011 
0.119 
0.139 
1.544 

12.978 

0.000 

0.463 

1.048 

4.599 

8.920 

0.818 

1.646 

18.531 


'Prob>F" 

1.0000 

0.8014 

0.2854 

0.0626 

0.6022 

0.0382 

0.6179 

0.5981 

0.2774 

0.7408 

0.3362 

0.7886 

0.8216 

0.7803 

0.7421 

0.8168 

0.0201 

0.2605 

0.2939 

0.7143 

0.7897 

0.8498 

0.5323 

0.1883 

0.0203 

0.9178 

0.7304 

0.7104 

0.2166 

0.0005 

0.9916 

0.4978 

0.3082 

0.0342 

0.0035 

0.3677 

0.2022 

0.0000 


Action 

Entered 

Entered 

Entered 

Entered 

Entered 

Entered 

Entered 

Entered 

Entered 

Removed 

Entered 

Entered 

Entered 


"Sig  Prob"  Seq  SS 

0.0117  12.00514 

0.0064  13.39518 

0.0325  7.888774 

0.0513  6.37973 

0.0112  10.44024 

0.0537  5.826821 

0.0838  4.586294 

0.0932  4.250829 

0.0499  5.682871 

0.2604  1 .8504 

0.2062  2.330128 

0.2170  2.21219 

0.2022  2.347732 


RSquare  Cp  p 

0.0513  19.001  2 

0.1086  12.669  3 

0.1424  9.7617  4 

0.1696  7.7932  5 

0.2143  3.2988  6 

0.2392  1.6742  7 

0.2588  0.8213  8 

0.2770  0.177  9 

0.3013  -1.358  10 

0.2934  -2.207  9 

0.3033  -1.656  10 

0.3128  -1.033  11 

0.3228  -0.493  12 
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Response  Column  59 
Actual  by  Predicted  Plot 


7-  ■ 


1-  ■■■ 


1 i — l - ' 1 — ' 1 — ' — I ' - 1 — ' 

1  2  3  4  5  6  7 

Column  59  Predicted  P<.0001  RSq=0.32 
RMSE=1 .1944 

Summary  of  Fit 


RSquare 

0.322832 

RSquare  Adj 

0.255726 

Root  Mean  Square  Error 

1.194424 

Mean  of  Response 

3.585366 

Observations  (or  Sum  Wgts) 

123 

Analysis  of  Variance 

Source  DF 

Sum  of  Squares 

Mean  Square 

F  Ratio 

Model  11 

75.49552 

6.86323 

4.8107 

Error  1 1 1 

158.35814 

1 .42665 

Prob  >  F 

C.  Total  122 

233.85366 

<.0001 

Lack  Of  Fit 

Source  DF 

Sum  of  Squares 

Mean  Square 

F  Ratio 

Lack  Of  Fit  43 

53.78591 

1.25084 

0.8134 

Pure  Error  68 

104.57222 

1.53783 

Prob  >  F 

Total  Error  111 

158.35814 

0.7640 

Max  RSq 

0.5528 

Parameter  Estimates 

Term 

Estimate 

Std  Error 

Intercept 

2.8586022 

0.34161 

whattypesystemscommunications[1-0] 

0.5083472 

0.270231 

whattypeshrinkbusiness[1  -0] 

0.5544567 

0.264271 

whattypeenterpriseOES[1  -0] 

2.0580277 

0.872661 

whatprocessreengineering[1  -0] 

0.3813011 

0.288021 

whatprocessappsupptjl  -0] 

0.6967357 

0.29601 

whatprocesscoding[1-0] 

-0.368866 

0.296814 

whatprocessfieldingjl  -0] 

-1.317295 

0.365658 

whatprocessnone[1  -0] 

-2.196161 

1 .024026 

whatproductscustom[1  -0] 

0.7770903 

0.260187 

whatproductscommoncust[1  -0] 

0.36664 

0.285808 

whatproductsnone[1  -0] 

2.5603306 

0.594765 

t  Ratio  Prob>|t| 

8.37  <.0001 

1 .88  0.0626 

2.10  0.0382 

2.36  0.0201 

1.32  0.1883 

2.35  0.0203 

-1.24  0.2166 

-3.60  0.0005 

-2.14  0.0342 

2.99  0.0035 

1.28  0.2022 

4.30  <.0001 


Effect  Tests 

Source 

whattypesystemscommunications 

whattypeshrinkbusiness 

whattypeenterpriseOES 

whatprocessreengineering 

whatprocessappsuppt 

whatprocesscoding 

whatprocessfielding 

whatprocessnone 

whatproductscustom 

whatproductscommoncust 

whatproductsnone 


Nparm  DF 

1  1 

1  1 

1  1 

1  1 

1  1 

1  1 

1  1 

1  1 

1  1 

1  1 

1  1 


Sum  of  Squares 
5.048573 
6.279905 
7.934660 
2.500370 
7.903873 
2.203369 
18.515375 
6.561818 
12.725925 
2.347732 
26.437376 


F  Ratio  Prob  >  F 

3.5388  0.0626 

4.4019  0.0382 

5.5617  0.0201 

1.7526  0.1883 

5.5402  0.0203 

1.5444  0.2166 

12.9782  0.0005 

4.5995  0.0342 

8.9201  0.0035 

1.6456  0.2022 

18.5311  <.0001 


Visibility  =  2.86  +  sys-comm(0.51)  +  shrink-bus(0.55)  +  ent-OES(2.06)  +  proc-reeng(0.38)  +  proc-appsup(0.70)  +  proc- 
coding(-0.37)  +  proc-field(-1.32)  +  proc-none(-2.20)  +  prod-cust(0.78)  +  prod-comcust(0.37)  +  prod-none(2.56) 
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Stepwise  Fit  -  Combined  Survey  Data  -  Consequences  (ControlProduct) 

Response: 

Column  59 


Stepwise  Regression  Control 

Prob  to  Enter  0.250 

Prob  to  Leave  0.250 

Direction: 

Rules: 


Current  Estimates 

SSE  DFE 

149.55216  114 


MSE  RSquare 

1.3118611  0.3039 


RSquare  Adj 
0.2429 


Cp  AIC 

-1.000899  44.41644 


Entered  Parameter 

X  Intercept 

whattypesystemsavionics{0-1} 

X  whattypesystemsembedded{0-1} 

whattypesystemscommunications{0-1} 
whattypesystemsdevice{0- 1 } 
whattypeshrinkbusiness{0-1} 

X  whattypeshrinkutilities{0-1} 

whattypeshrinkinternet{0-1 } 

X  whattypecomponentdomain{0-1} 

whattypecomponentCASE{0-1} 
whattypecomponentclass{0- 1 } 
whattypecomponentOS{0-1 } 
whattypecomponentdevelopment{0-1} 
whattypeenterpriseacctng{0-1} 
whattypeenterprisemanufact{0-1 } 
whattypeenterprisepayroll{0-1} 
whattypeenterpriseOES{0- 1 } 
whattypeenterprisescripting{0-1} 
whattypeenterpriseweb{0-1 } 
whatprocessrequirements{0-1} 
whatprocessdesign{0- 1 } 
whatprocesstesting{0-1} 
whatprocessmaintenance{0-1} 

X  whatprocessreengineering{0-1} 

X  whatprocessappsuppt{0-1} 

whatprocesstraining{0-1} 
whatprocessspecification{0-1 } 

X  whatprocessdocumentation{0-1} 

whatprocesscoding{0- 1 } 

X  whatprocessfielding{0-1} 

whatprocessCM{0-1} 

X  whatprocesstoolsuppt{0-1} 

whatprocessS  WEngSuppt{0- 1 } 
whatprocessnone{0-1 } 
whatproductscustom{0-1 } 

X  whatproductsCOTS{0-1} 

whatproductscommoncust{0-1} 
whatproductsnone{0-1 } 


Step  History 

Step  Parameter 


9 

10 


whatprocessfielding{0-1 } 
whatprocessappsuppt{0-1 } 
whatproductsnone{0-1} 
whatprocessreengineering{0-1 } 
whatproductsCOTS{0-1} 
whattypeshrinkutilities{0-1} 
whatprocesstoolsuppt{0-1} 
whattypecomponentdomain{0-1 } 
whattypesystemsembedded{0-1 } 
whatprocessdocumentation{0-1} 


Action 

Entered 

Entered 

Entered 

Entered 

Entered 

Entered 

Entered 

Entered 

Entered 

Entered 


Estimate  nDF 

4.39807857  1 

0  1 

0.27157371  1 

0  1 

0  1 

0  1 

-0.8418239  1 

0  1 

0.39122876  1 

0  1 

0  1 

0  1 

0  1 

0  1 

0  1 

0  1 

0  1 

0  1 

0  1 

0  1 

0  1 

0  1 

0  1 

-0.4178804  1 

-0.4136855  1 

0  1 

0  1 

-0.2246851  1 

0  1 

0.5839269  1 

0  1 

0.24238327  1 

0  1 

0  1 

0  1 

0.40089332  1 

0  1 

-0.4394939  1 


"Sig  Prob" 
0.0136 
0.0022 
0.0254 
0.0310 
0.0478 
0.0234 
0.0804 
0.1059 
0.1828 
0.2097 


Seq  SS 
10.41445 
15.17643 
7.684275 
6.935124 
5.680442 
7.232595 
4.189049 
3.526401 
2.369105 
2.087969 


SS 

0 

0.781292 

3.401116 

1.150687 

0.408255 

0.049642 

11.6627 

0.618538 

4.070965 

0.305409 

0.002439 

I. 50076 
0.210125 
1 .090822 
1.035315 
0.591976 
0.220122 
0.048012 

1.02201 

0.668217 

0.279946 

0.709319 

1.378113 

II. 5409 
10.46595 
1.101103 

0.21449 

2.087969 

0.052207 

13.73191 

1.379668 

3.645529 

0.773598 

0.08412 

0.719046 

10.74864 

1.477006 

4.861569 


RSquare 

0.0485 

0.1191 

0.1549 

0.1872 

0.2136 

0.2473 

0.2668 

0.2832 

0.2942 

0.3039 


"F  Ratio"  "Prob>F" 

0.000  1.0000 

0.593  0.4427 

2.593  0.1101 

0.876  0.3512 

0.309  0.5792 

0.038  0.8468 

8.890  0.0035 

0.469  0.4947 

3.103  0.0808 

0.231  0.6315 

0.002  0.9658 

1.145  0.2868 

0.159  0.6908 

0.830  0.3641 

0.788  0.3767 

0.449  0.5041 

0.167  0.6840 

0.036  0.8493 

0.778  0.3798 

0.507  0.4778 

0.212  0.6462 

0.539  0.4646 

1.051  0.3075 

8.797  0.0037 

7.978  0.0056 

0.838  0.3619 

0.162  0.6878 

1.592  0.2097 

0.039  0.8429 

10.468  0.0016 

1.052  0.3072 

2.779  0.0983 

0.588  0.4450 

0.064  0.8014 

0.546  0.4615 

8.193  0.0050 

1.127  0.2906 

3.706  0.0567 


Cp  p 

18.43  2 

10.079  3 

6.8382  4 

4.1082  5 

2.234  6 

-0.699  7 

-1 .556  8 

-1.961  9 

-1.577  10 

-1.001  11 
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Response  Column  59 
Actual  by  Predicted  Plot 


I 


1  2  3  4  5  6  7 

Column  59  Predicted  P<.0001  RSq=0.30 
RMSE=1 .1454 


Summary  of  Fit 


RSquare  0.303916 

RSquare  Adj  0.242856 

Root  Mean  Square  Error  1.145365 

Mean  of  Response  3.904 

Observations  (or  Sum  Wgts)  125 


Analysis  of  Variance 


Source  DF 

Model  10 

Error  114 

C.  Total  124 


Sum  of  Squares 
65.29584 
149.55216 
214.84800 


Mean  Square  F  Ratio 

6.52958  4.9773 

1.31186  Prob  >  F 

<.0001 


Lack  Of  Fit 


Source  DF 

Lack  Of  Fit  36 

Pure  Error  78 

Total  Error  114 


Sum  of  Squares 
52.03764 
97.51452 
149.55216 


Mean  Square  F  Ratio 

1.44549  1.1562 

1.25019  Prob  >  F 

0.2924 
Max  RSq 
0.5461 


Parameter  Estimates 

Term 

Intercept 

whattypesystemsembeddedjl  -0] 
whattypeshrinkutilities[1  -0] 
whattypecomponentdomain[1  -0] 
whatprocessreengineering[1-0] 
whatprocessappsuppt[1-0] 
whatprocessdocumentationjl  -0] 
whatprocessfieldingjl  -0] 
whatprocesstoolsuppt[1  -0] 
whatproductsCOTSjl  -0] 
whatproductsnone[1  -0] 


Estimate 

3.9505157 

-0.543147 

1.6836479 

-0.782458 

0.8357607 

0.8273711 

0.4493702 

-1.167854 

-0.484767 

-0.801787 

0.8789878 


Std  Error 
0.149943 
0.337327 
0.564671 
0.444177 
0.281777 
0.292924 
0.356194 
0.360966 
0.290801 
0.280108 
0.456603 


t  Ratio 
26.35 
-1.61 
2.98 
-1.76 
2.97 
2.82 
1.26 
-3.24 
-1.67 
-2.86 
1.93 


Prob>|t| 

<.0001 

0.1101 

0.0035 

0.0808 

0.0037 

0.0056 

0.2097 

0.0016 

0.0983 

0.0050 

0.0567 


Effect  Tests 

Source 

whattypesystemsembedded 

whattypeshrinkutilities 

whattypecomponentdomain 

whatprocessreengineering 

whatprocessappsuppt 

whatprocessdocumentation 

whatprocessfielding 

whatprocesstoolsuppt 

whatproductsCOTS 

whatproductsnone 


Nparm  DF 

1  1 

1  1 

1  1 

1  1 

1  1 

1  1 

1  1 

1  1 

1  1 

1  1 


Sum  of  Squares 
3.401116 
11.662705 
4.070965 
11.540897 
10.465953 
2.087969 
13.731912 
3.645529 
10.748639 
4.861569 


F  Ratio  Prob  >  F 

2.5926  0.1101 

8.8902  0.0035 

3.1032  0.0808 

8.7973  0.0037 

7.9779  0.0056 

1.5916  0.2097 

10.4675  0.0016 

2.7789  0.0983 

8.1934  0.0050 

3.7059  0.0567 


ControlProduct  =  3.95  +  sys-embed(-0.54)  +  shrink-util(1 .68)  +  comp-domain(-0.78)  +  proc-reeng(0.84)  +  proc- 
appsup(0.83)  +  proc-doc(0.45)  +  proc-field(-1.17)  +  proc_toolsup(-0.48)  +  prod-COTS(-0.80)  +  prod-none(0.88) 
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Stepwise  Fit  -  Combined  Survey  Data  -  Consequences  (ChangeCost) 

Response: 

Column  59 

Stepwise  Regression  Control 

Prob  to  Enter  0.250 

Prob  to  Leave  0.250 

Direction: 

Rules: 


Current  Estimates 

SSE  DFE 

182.1889  109 


MSE  RSquare 

1.6714578  0.2761 


RSquare  Adj 
0.1831 


Cp  AIC 

1.0826579  77.71055 


Entered  Parameter 

X  Intercept 

X  whattypesystemsavionics{0-1} 

whattypesystemsembedded{0-1 } 
whattypesystemscommunications{0-1} 
whattypesystemsdevice{0-1} 

X  whattypeshrinkbusiness{0-1} 

whattypeshrinkutilities{0-1} 

X  whattypeshrinkinternet{0-1} 

whattypecomponentdomain{0-1} 

whattypecomponentCASE{0-1} 

X  whattypecomponentclass{0-1} 

X  whattypecomponentOS{0-1} 

whattypecomponentdevelopment{0-1} 

whattypeenterpriseacctng{0-1} 

X  whattypeenterprisemanufact{0-1} 

whattypeenterprisepayroll{0-1} 
whattypeenterpriseOES{0- 1 } 
whattypeenterprisescripting{0-1} 
whattypeenterpriseweb{0-1 } 
whatprocessrequirements{0-1} 

X  whatprocessdesign{0-1} 

X  whatprocesstesting{0-1} 

whatprocessmaintenance{0-1} 
whatprocessreengineering{0-1 } 
whatprocessappsuppt{0-1 } 
whatprocesstraining{0-1} 

X  whatprocessspecification{0-1} 

whatprocessdocumentation{0-1 } 
whatprocesscoding{0- 1 } 
whatprocessfielding{0-1} 

X  whatprocessCM{0-1} 

whatprocesstoolsuppt{0-1} 
whatprocessSWEngSuppt{0- 1 } 

X  whatprocessnone{0-1} 

whatproductscustom{0-1 } 

X  whatproductsCOTS{0-1} 

X  whatproductscommoncust{0-1} 

X  whatproductsnone{0-1} 


Step  History 

Step  Parameter 


9 

10 

11 

12 

13 

14 


whatproductsnone{0-1 } 
whattypecomponentclass{0-1 } 
whatproductscommoncust{0-1} 
whatprocesstesti  ng{0- 1 } 
whattypesystemsavionics{0-1} 
whatprocessspecification{0-1 } 
whattypeenterprisemanufact{0-1} 
whattypeshrinkbusiness{0-1 } 
whatprocessnone{0-1 } 
whattypecomponentOS{0-1 } 
whatprocessdesign{0-1 } 
whatprocessCM{0-1} 
whatproductsCOTS{0-1} 
whattypeshrinkinternet{0-1} 


Action 

Entered 

Entered 

Entered 

Entered 

Entered 

Entered 

Entered 

Entered 

Entered 

Entered 

Entered 

Entered 

Entered 

Entered 


Estimate  nDF 

5.27126411  1 

-0.5939639  1 

0  1 

0  1 

0  1 

0.23493251  1 

0  1 

0.24817787  1 

0  1 

0  1 

0.63733259  1 

-0.3885275  1 

0  1 

0  1 

-0.6949918  1 

0  1 

0  1 

0  1 

0  1 

0  1 

-0.230211  1 

0.32425223  1 

0  1 

0  1 

0  1 

0  1 

-0.331706  1 

0  1 

0  1 

0  1 

0.27679138  1 

0  1 

0  1 

-0.7388523  1 

0  1 

-0.1975663  1 

-0.3805012  1 

0.67920278  1 


SS 

0 

7.716578 
0.719231 
0.523547 
0.039326 
4.200341 
0.083262 
2.58409 
0.51206 
1.594578 
6.705303 
4.236056 
1.31359 
1.351273 
6.883628 
1 .653086 
1 .563605 
0.872628 
0.245861 
0.830499 
2.986489 
7.071045 
0.49454 
0.320418 
0.098376 
0.868856 
7.468708 
0.920031 
0.04385 
0.018616 
4.573154 
1.670843 
2.059798 
3.041814 
0.360992 
2.787146 
10.24175 
7.941852 


"Sig  Prob" 
0.0086 
0.0317 
0.0469 
0.1274 
0.1138 
0.0953 
0.1405 
0.1588 
0.1537 
0.1842 
0.2153 
0.1426 
0.2305 
0.2164 


Seq  SS 
13.8972 
8.932005 
7.438187 
4.30053 
4.572401 
5.015248 
3.865907 
3.500871 
3.556793 
3.057907 
2.647287 
3.677618 
2.442474 
2.58409 


RSquare 

0.0552 

0.0907 

0.1203 

0.1374 

0.1555 

0.1754 

0.1908 

0.2047 

0.2188 

0.2310 

0.2415 

0.2561 

0.2658 

0.2761 


"F  Ratio"  "Prob>F" 

0.000  1.0000 

4.617  0.0339 

0.428  0.5143 

0.311  0.5781 

0.023  0.8789 

2.513  0.1158 

0.049  0.8246 

1.546  0.2164 

0.304  0.5823 

0.954  0.3310 

4.012  0.0477 

2.534  0.1143 

0.784  0.3778 

0.807  0.3710 

4.118  0.0449 

0.989  0.3222 

0.935  0.3357 

0.520  0.4725 

0.146  0.7032 

0.495  0.4834 

1.787  0.1841 

4.230  0.0421 

0.294  0.5888 

0.190  0.6636 

0.058  0.8096 

0.518  0.4735 

4.468  0.0368 

0.548  0.4607 

0.026  0.8722 

0.011  0.9165 

2.736  0.1010 

1.000  0.3196 

1.235  0.2689 

1.820  0.1801 

0.214  0.6443 

1.667  0.1993 

6.127  0.0148 

4.751  0.0314 


Cp  p 

4.0952  2 

1.4337  3 

-0.448  4 

-0.693  5 

-1.079  6 

-1.696  7 

-1.714  8 

-1.541  9 

-1.397  10 

-0.993  1 1 

-0.375  12 

-0.294  13 

0.4313  14 

1.0827  15 


Appendix  D 


Response  Column  59 
Actual  by  Predicted  Plot 


Summary  of  Fit 


RSquare 
RSquare  Adj 
Root  Mean  Square  Error 
Mean  of  Response 
Observations  (or  Sum  Wgts) 
Analysis  of  Variance 
Source  DF 

Model  14 

Error  109 

C.  Total  123 

Lack  Of  Fit 

Source  DF 

Lack  Of  Fit  48 

Pure  Error  61 

Total  Error  109 


0.276102 

0.183124 

1.292849 

4.225806 

124 


Sum  of  Squares 
69.48852 
182.18890 
251.67742 


Sum  of  Squares 
98.63890 
83.55000 
182.18890 


Parameter  Estimates 

Term 

Intercept 

whattypesystemsavionics[1  -0] 
whattypeshrinkbusiness[1  -0] 
whattypeshrinkinternet[1  -0] 
whattypecomponentclass[1  -0] 
whattypecomponentOS[1  -0] 
whattypeenterprisemanufact[1  -0] 
whatprocessdesign[1  -0] 
whatprocesstesting[1  -0] 
whatprocessspecification[1  -0] 
whatprocessCM[1  -0] 
whatprocessnone[1  -0] 
whatproductsCOTS[1-0] 
whatproductscommoncust[1  -0] 


whatproductsnone[1-0] 

Effect  Tests 

Source  Nparm 

whattypesystemsavionics  1 

whattypeshrinkbusiness  1 

whattypeshrinkinternet  1 

whattypecomponentclass  1 

whattypecomponentOS  1 

whattypeenterprisemanufact  1 

whatprocessdesign  1 

whatprocesstesting  1 

whatprocessspecification  1 

whatprocessCM  1 

whatprocessnone  1 

whatproductsCOTS  1 

whatproductscommoncust  1 

whatproductsnone  1 


Mean  Square  F  Ratio 

4.96347  2.9695 

1.67146  Prob  >  F 

0.0007 


Mean  Square  F  Ratio 

2.05498  1.5003 

1.36967  Prob  >  F 

0.0671 
Max  RSq 
0.6680 


Estimate 
4.1156334 
1.1879278 
-0.469865 
-0.496356 
-1.274665 
0.7770549 
1.3899836 
0.460422 
-0.648504 
0.6634121 
-0.553583 
1.4777047 
0.3951327 
0.7610024 
-1 .358406 


Std  Error 
0.29475 
0.552873 
0.2964 
0.399197 
0.636407 
0.488111 
0.684934 
0.344448 
0.315296 
0.31384 
0.334674 
1.095391 
0.305993 
0.30743 
0.623184 


t  Ratio  Prob>|t| 

13.96  <.0001 

2.15  0.0339 

-1.59  0.1158 

-1.24  0.2164 

-2.00  0.0477 

1.59  0.1143 

2.03  0.0449 

1.34  0.1841 

-2.06  0.0421 

2.11  0.0368 

-1.65  0.1010 

1.35  0.1801 

1.29  0.1993 

2.48  0.0148 

-2.18  0.0314 


DF  Sum  of  Squares 

1  7.716578 

1  4.200341 

1  2.584090 

1  6.705303 

1  4.236056 

1  6.883628 

1  2.986489 

1  7.071045 

1  7.468708 

1  4.573154 

1  3.041814 

1  2.787146 

1  10.241752 

1  7.941852 


F  Ratio  Prob  >  F 

4.6167  0.0339 

2.5130  0.1158 

1.5460  0.2164 

4.0116  0.0477 

2.5343  0.1143 

4.1183  0.0449 

1.7868  0.1841 

4.2305  0.0421 

4.4684  0.0368 

2.7360  0.1010 

1.8199  0.1801 

1.6675  0.1993 

6.1274  0.0148 

4.7515  0.0314 


ChangeCost  =  4.12  +  sys-avia(1.19)  +  shrink-bus(-0.47)  +  shrink-int(-0.50)  +  comp-class(-1 .27)  +  comp-OS(0.78)  +  ent- 
mnft(1.39)  +  proc-des(0.46)  +  proc-test(-0.65)  +  proc-spec(0.66)  +  proc-CM(-0.55)  +  proc-none(1.48)  +  prod-COTS(0.40) 
+  prod-comcust(0.76)  +  prod-none(-1.36) 
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Stepwise  Fit  -  Combined  Survey  Data  -  Consequences  (LangCulture) 

Response: 

Column  59 


Stepwise  Regression  Control 

Prob  to  Enter  0.250 

Prob  to  Leave  0.250 

Direction: 

Rules: 


Current  Estimates 

SSE  DFE 

85.614974  110 


MSE  RSquare 

0.7783179  0.2228 


RSquare  Adj 
0.1451 


Cp 

-3.851436 


AIC 

-19.2076 


Lock 

X 


Entered  Parameter 

X  Intercept 

X  whattypesystemsavionics{0-1} 

X  whattypesystemsembedded{0-1} 

whattypesystemscommunications{0-1} 
whattypesystemsdevice{0- 1 } 
whattypeshrinkbusiness{0-1} 
whattypeshrinkutilities{0-1 } 
whattypeshrinkinternet{0-1 } 
whattypecomponentdomain{0-1} 
whattypecomponentCASE{0-1} 

X  whattypecomponentclass{0-1} 

X  whattypecomponentOS{0-1} 

X  whattypecomponentdevelopment{0-1} 

X  whattypeenterpriseacctng{0-1} 

whattypeenterprisemanufact{0-1 } 

X  whattypeenterprisepayroll{0-1} 

whattypeenterpriseOES{0- 1 } 
whattypeenterprisescripting{0-1} 
whattypeenterpriseweb{0-1 } 
whatprocessrequirements{0-1} 
whatprocessdesign{0- 1 } 
whatprocesstesting{0- 1 } 
whatprocessmaintenance{0-1} 
whatprocessreengineering{0-1} 

X  whatprocessappsuppt{0-1} 

whatprocesstraining{0-1} 
whatprocessspecification{0-1 } 
whatprocessdocumentation{0-1 } 
whatprocesscoding{0- 1 } 
whatprocessfielding{0-1} 
whatprocessCM{0-1} 
whatprocesstoolsuppt{0-1} 

X  whatprocessSWEngSuppt{0-1} 

whatprocessnone{0-1 } 
whatproductscustom{0-1} 

X  whatproductsCOTS{0-1} 

X  whatproductscommoncust{0-1} 

whatproductsnone{0-1 } 


Step  History 

Step  Parameter 

1  whatprocessappsuppt{0-1} 

2  whattypecomponentOS{0-1} 

3  whatproductscommoncust{0-1} 

4  whattypeshrinkbusiness{0-1} 

5  whattypesystemsavionics{0-1} 

6  whattypecomponentdevelopment{0- 1 } 

7  whattypesystemsembedded{0-1} 

8  whattypeshrinkbusiness{0-1} 

9  whatprocessmaintenance{0-1} 

10  whattypeenterprisepayroll{0-1} 

1 1  whattypecomponentclass{0-1 } 

12  whatprocessSWEngSuppt{0-1} 

13  whatproductsCOTS{0-1} 

14  whattypeenterpriseacctng{0-1} 

15  whatprocessmaintenance{0-1} 


Estimate  nDF 

4.6380565  1 

-0.3500808  1 

-0.2931314  1 

0  1 

0  1 

0  1 

0  1 

0  1 

0  1 

0  1 

0.63799671  1 

0.37983132  1 

0.14034206  1 

-0.2070367  1 

0  1 

-0.5678941  1 

0  1 

0  1 

0  1 

0  1 

0  1 

0  1 

0  1 

0  1 

0.25714825  1 

0  1 

0  1 

0  1 

0  1 

0  1 

0  1 

0  1 

-0.1463898  1 

0  1 

0  1 

-0.1588631  1 

-0.2182791  1 

0  1 


SS 

0 

2.948675 
3.846151 
0.021928 
0.062169 
0.359798 
0.440956 
0.000432 
0.169986 
0.40814 
4.917043 
3.966181 
1.298713 
1 .270391 
0.439016 
5.056645 
0.11079 
0.236924 
0.313628 
0.956486 
0.173837 
0.289753 
0.917058 
0.014185 
3.893406 
1.002621 
0.082599 
0.132814 
0.099159 
0.686525 
0.000019 
0.005267 
1.711346 
0.026231 
0.483697 
1.736235 
3.597185 
0.535693 


"F  Ratio" 
0.000 
3.789 
4.942 
0.028 
0.079 
0.460 
0.564 
0.001 
0.217 
0.522 
6.318 
5.096 
1.669 
1.632 
0.562 
6.497 
0.141 
0.302 
0.401 
1.232 
0.222 
0.370 
1.180 
0.018 
5.002 
1.292 
0.105 
0.169 
0.126 
0.881 
0.000 
0.007 
2.199 
0.033 
0.619 
2.231 
4.622 
0.686 


'Prob>F" 

1.0000 

0.0542 

0.0283 

0.8676 

0.7789 

0.4991 

0.4541 

0.9813 

0.6424 

0.4715 

0.0134 

0.0260 

0.1992 

0.2041 

0.4551 

0.0122 

0.7078 

0.5835 

0.5280 

0.2696 

0.6386 

0.5442 

0.2797 

0.8933 

0.0273 

0.2582 

0.7462 

0.6815 

0.7229 

0.3500 

0.9961 

0.9349 

0.1410 

0.8553 

0.4330 

0.1382 

0.0338 

0.4092 


Action 

Entered 

Entered 

Entered 

Entered 

Entered 

Entered 

Entered 

Removed 

Entered 

Entered 

Entered 

Entered 

Entered 

Entered 

Removed 


'Sig  Prob"  Seq  SS 

0.0426  3.726009 

0.0649  3.017563 

0.0830  2.612083 

0.1210  2.059673 

0.2040  1.369954 

0.2076  1.341197 

0.2101  1.31984 

0.2535  1.09421 

0.1183  2.037389 

0.1837  1.461845 

0.0269  3.966099 

0.2098  1.24858 

0.1956  1.322177 

0.2415  1.077821 

0.2797  0.917058 


RSquare  Cp  p 

0.0338  -0.953  2 

0.0612  -2.271  3 

0.0849  -3.144  4 

0.1036  -3.409  5 

0.1161  -2.915  6 

0.1282  -2.39  7 

0.1402  -1.841  8 

0.1303  -2.638  7 

0.1488  -2.879  8 

0.1620  -2.486  9 

0.1980  -4.848  10 

0.2094  -4.221  11 

0.2214  -3.675  12 

0.2312  -2.86  13 

0.2228  -3.851  12 
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Response  Column  59 
Actual  by  Predicted  Plot 


1  2  3  4  5  6  7 

Column  59  Predicted  P=0.0024  RSq=0.22 
RMSE=0.8822 


Summary  of  Fit 


RSquare  0.22284 

RSquare  Adj  0.145124 

Root  Mean  Square  Error  0.882223 

Mean  of  Response  4.31 1475 

Observations  (or  Sum  Wgts)  122 


Analysis  of  Variance 

Source 

DF 

Sum  of  Squares 

Mean  Square 

Model 

11 

24.54896 

2.23172 

Error 

110 

85.61497 

0.77832 

C.  Total 

121 

110.16393 

F  Ratio 
2.8674 
Prob  >  F 
0.0024 


Lack  Of  Fit 


Source  DF 

Lack  Of  Fit  33 

Pure  Error  77 

Total  Error  110 


Sum  of  Squares 
29.068942 
56.546032 
85.614974 


Mean  Square 
0.880877 
0.734364 


F  Ratio 
1.1995 
Prob  >  F 
0.2541 
Max  RSq 
0.4867 


Parameter  Estimates 

Term 

Intercept 

whattypesystemsavionics[1  -0] 

whattypesystemsembedded[1-0] 

whattypecomponentclass[1  -0] 

whattypecomponentOS[1  -0] 

whattypecomponentdevelopment[1-0] 

whattypeenterpriseacctng[1-0] 

whattypeenterprisepayroll[1-0] 

whatprocessappsuppt[1-0] 

whatprocessSWEngSuppt[1  -0] 

whatproductsCOTS[1-0] 

whatproductscommoncust[1  -0] 


Estimate 
4.1116998 
0.7001615 
0.5862629 
-1 .275993 
-0.759663 
-0.280684 
0.4140734 
1.1357881 
-0.514297 
0.2927796 
0.3177263 
0.4365582 


Std  Error  t  Ratio  Prob>|t| 
0.137641  29.87  <.0001 

0.359719  1.95  0.0542 

0.263729  2.22  0.0283 

0.507662  -2.51  0.0134 

0.336522  -2.26  0.0260 

0.21729  -1.29  0.1992 

0.324106  1.28  0.2041 

0.445599  2.55  0.0122 

0.229947  -2.24  0.0273 

0.197447  1.48  0.1410 

0.212729  1.49  0.1382 

0.203067  2.15  0.0338 


Effect  Tests 

Source 

whattypesystemsavionics 

whattypesystemsembedded 

whattypecomponentclass 

whattypecomponentOS 

whattypecomponentdevelopment 

whattypeenterpriseacctng 

whattypeenterprisepayroll 

whatprocessappsuppt 

whatprocessSWEngSuppt 

wh  atprod  uctsCOT  S 

whatproductscommoncust 


Nparm  DF 

1  1 

1  1 

1  1 

1  1 

1  1 

1  1 

1  1 

1  1 

1  1 

1  1 

1  1 


Sum  of  Squares 
2.9486750 
3.8461511 
4.9170425 
3.9661810 
1.2987132 
1.2703910 
5.0566448 
3.8934065 
1.7113455 
1.7362352 
3.5971846 


F  Ratio  Prob  >  F 

3.7885  0.0542 

4.9416  0.0283 

6.3175  0.0134 

5.0958  0.0260 

1.6686  0.1992 

1.6322  0.2041 

6.4969  0.0122 

5.0023  0.0273 

2.1988  0.1410 

2.2308  0.1382 

4.6217  0.0338 


LangCult  =  4.11  +  sys-avia(0.70)  +  sys-embed(0.59)  +  comp-class(-1 .28)  +  comp-OS(-0.76)  +  comp-dev(-0.28)  +  ent- 
acct(0.41)  +  ent-pay(1.14)  +  proc-appsup(-0.51)  +  proc+SWEngSup(0.29)  +  prod-COTS(0.32)  +  prod-comcust(0.44) 
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Stepwise  Fit  -  Combined  Survey  Data  -  Consequences  (TurfWar) 

Response: 

Column  59 

Stepwise  Regression  Control 

Prob  to  Enter  0.250 

Prob  to  Leave  0.250 

Direction: 

Rules: 


Current  Estimates 

SSE  DFE 

137.06985  105 


MSE  RSquare 

1.3054272  0.3554 


RSquare  Adj  Cp  AIC 

0.2572  4.4366712  48.20929 


Lock 

X 


X  Intercept 

whattypesystemsavionics{0-1} 

X  whattypesystemsembedded{0-1} 

whattypesystemscommunications{0-1} 
whattypesystemsdevice{0-1} 
whattypeshrinkbusiness{0-1} 
whattypeshrinkutilities{0-1 } 
whattypeshrinkinternet{0-1 } 
whattypecomponentdomain{0-1} 

X  whattypecomponentCASE{0-1} 

whattypecomponentclass{0- 1 } 

X  whattypecomponentOS{0-1} 

X  whattypecomponentdevelopment{0-1} 

X  whattypeenterpriseacctng{0-1} 

X  whattypeenterprisemanufact{0-1} 

whattypeenterprisepayroll{0-1 } 
whattypeenterpriseOES{0- 1 } 
whattypeenterprisescripting{0-1} 

X  whattypeenterpriseweb{0-1} 

X  whatprocessrequirements{0-1} 

whatprocessdesign{0- 1 } 
whatprocesstesting{0- 1 } 
whatprocessmaintenance{0-1} 

X  whatprocessreengineering{0-1) 

X  whatprocessappsuppt{0-1} 

X  whatprocesstraining{0-1} 

whatprocessspecification{0-1 } 
whatprocessdocumentation{0-1 } 

X  whatprocesscoding{0-1} 

whatprocessfielding{0-1} 

X  whatprocessCM{0-1} 

X  whatprocesstoolsuppt{0-1} 

whatprocessSWEngSuppt{0- 1 } 

X  whatprocessnone{0-1} 

whatproductscustom{0-1 } 

X  whatproductsCOTS{0-1} 

whatproductscommoncust{0-1} 
whatproductsnone{0-1 } 


Step  History 

Step  Parameter 

1  whatprocessappsuppt{0-1} 

2  whattypeenterprisemanufact{0-1} 

3  whatproductsCOTS{0-1} 

4  whatprocesstoolsuppt{0-1} 

5  whatprocessreengineering{0-1} 

6  whattypecomponentOS{0-1} 

7  whattypesystemsembedded{0-1} 

8  whattypeenterpriseacctng{0-1} 

9  whatprocessSWEngSuppt{0-1} 

1 0  whattypecomponentdevelopment{0- 1 } 

11  whatprocesscoding{0-1} 

12  whatprocesstraining{0-1} 

13  whatprocessrequirements{0-1} 

14  whatprocessSWEngSuppt{0-1} 

15  whatprocessnone{0-1} 

16  whattypecomponentCASE{0-1} 

17  whatprocessCM{0-1} 

18  whattypeenterpriseweb{0-1} 


Estimate  nDF 

4.56688949  1 

0  1 

-0.2365855  1 

0  1 

0  1 

0  1 

0  1 

0  1 

0  1 

0.52026112  1 

0  1 

0.53528226  1 

-0.2394009  1 

-0.3997281  1 

-0.7099058  1 

0  1 

0  1 

0  1 

-0.3153353  1 

-0.2686756  1 

0  1 

0  1 

0  1 

0.48676186  1 

0.57408812  1 

0.34172417  1 

0  1 

0  1 

0.30232284  1 

0  1 

0.25408779  1 

-0.7733196  1 

0  1 

0.55882584  1 

0  1 

-0.5796411  1 

0  1 

0  1 


SS 

0 

0.020697 
2.419592 
0.363862 
0.258886 
0.486793 
0.003339 
0.013964 
0.000435 
4.303907 
0.364638 
7.571654 
3.326867 
4.524383 
6.621548 
0.784754 
0.639765 
0.337333 
3.026957 
5.280131 
0.356763 
0.914068 
0.00006 
11.6541 
17.99557 
5.76081 1 
0.564068 
0.333817 
5.291409 
0.737188 
3.142885 
23.71521 
1.461392 
2.22603 
0.044254 
20.46879 
0.466721 
0.000268 


"F  Ratio" 
0.000 
0.016 
1.853 
0.277 
0.197 
0.371 
0.003 
0.011 
0.000 
3.297 
0.277 
5.800 
2.548 
3.466 
5.072 
0.599 
0.488 
0.257 
2.319 
4.045 
0.271 
0.698 
0.000 
8.927 
13.785 
4.413 
0.430 
0.254 
4.053 
0.562 
2.408 
18.167 
1.121 
1.705 
0.034 
15.680 
0.355 
0.000 


'Prob>F" 

1.0000 

0.9005 

0.1763 

0.5999 

0.6582 

0.5440 

0.9600 

0.9182 

0.9855 

0.0723 

0.5995 

0.0178 

0.1134 

0.0654 

0.0264 

0.4408 

0.4865 

0.6136 

0.1308 

0.0469 

0.6035 

0.4053 

0.9946 

0.0035 

0.0003 

0.0381 

0.5136 

0.6154 

0.0466 

0.4550 

0.1238 

0.0000 

0.2922 

0.1945 

0.8549 

0.0001 

0.5524 

0.9887 


Action 

Entered 

Entered 

Entered 

Entered 

Entered 

Entered 

Entered 

Entered 

Entered 

Entered 

Entered 

Entered 

Entered 

Removed 

Entered 

Entered 

Entered 

Entered 


'Sig  Prob"  Seq  SS 

0.0092  11.74757 

0.0303  7.799643 

0.0283  7.748843 

0.0230  8.046927 

0.0526  5.674788 

0.1010  3.985615 

0.1233  3.471789 

0.1012  3.873055 

0.1548  2.88453 

0.1545  2.862171 

0.1077  3.610261 

0.1715  2.579441 

0.1513  2.814811 

0.3401  1.237236 

0.2098  2.131854 

0.1983  2.230599 

0.1867  2.334264 

0.1308  3.026957 


RSquare  Cp  p 

0.0552  17.488  2 

0.0919  14.228  3 

0.1284  11.002  4 

0.1662  7.5755  5 

0.1929  5.7486  6 

0.2116  5.0608  7 

0.2280  4.7195  8 

0.2462  4.1076  9 

0.2597  4.1623  10 

0.2732  4.2322  11 

0.2902  3.7975  12 

0.3023  4.058  13 

0.3155  4.1597  14 

0.3097  2.9941  13 

0.3197  3.5564  14 

0.3302  4.0522  15 

0.3412  4.478  16 

0.3554  4.4367  17 
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Response  Column  59 
Actual  by  Predicted  Plot 


Column  59  Predicted  P<.0001  RSq=0.36 
RMSE=1 .1426 


Summary  of  Fit 


RSquare 
RSquare  Adj 
Root  Mean  Square  Error 
Mean  of  Response 
Observations  (or  Sum  Wgts) 
Analysis  of  Variance 


Source  DF 

Model  16 

Error  1 05 

C.  Total  121 


Parameter  Estimates 

Term 
Intercept 

whattypesystemsembedded[1  -0] 
whattypecomponentCASE[1  -0] 
whattypecomponentOS[1  -0] 
whattypecomponentdevelopment[1-0] 
whattypeenterpriseacctng[1  -0] 
whattypeenterprisemanufact[1  -0] 
whattypeenterpriseweb[1-0] 
whatprocessrequirements[1  -0] 
whatprocessreengineering[1  -0] 
whatprocessappsuppt[1  -0] 
whatprocesstraining[1  -0] 
whatprocesscoding[1  -0] 
whatprocessCM[1-0] 
whatprocesstoolsuppt[1  -0] 
whatprocessnone[1  -0] 
whatproductsCOTS[1  -0] 

Effect  Tests 

Source 

whattypesystemsembedded 

whattypecomponentCASE 

whattypecomponentOS 

whattypecomponentdevelopment 

whattypeenterpriseacctng 

whattypeenterprisemanufact 

whattypeenterpriseweb 

whatprocessrequirements 

whatprocessreengineering 

whatprocessappsuppt 

whatprocesstraining 

whatprocesscoding 

whatprocessCM 

whatprocesstoolsuppt 

whatprocessnone 

whatproductsCOTS 


0.355438 

0.257219 

1.142553 

4.377049 


Sum  of  Squares  Mean  Square 

75.58589  4.72412 

137.06985  1.30543 

212.65574 


Estimate 
4.6176517 
0.473171 
-1 .040522 
-1.070565 
0.4788018 
0.7994561 
1.4198116 
0.6306706 
0.5373511 
-0.973524 
-1.148176 
-0.683448 
-0.604646 
-0.508176 
1.5466391 
-1.117652 
1.1592822 


Nparm  DF 

1  1 

1  1 

1  1 

1  1 

1  1 

1  1 

1  1 

1  1 

1  1 

1  1 

1  1 

1  1 

1  1 

1  1 

1  1 

1  1 


F  Ratio 
3.6188 
Prob  >  F 
<.0001 


Std  Error 
0.282545 
0.347555 
0.573055 
0.444523 
0.299926 
0.429429 
0.630416 
0.414167 
0.267185 
0.325824 
0.309245 
0.325342 
0.300325 
0.32751 1 
0.362871 
0.855889 
0.292765 


t  Ratio 
16.34 
1.36 
-1.82 
-2.41 
1.60 
1.86 

2.25 
1.52 
2.01 
-2.99 
-3.71 
-2.10 
-2.01 
-1.55 

4.26 
-1.31 
3.96 


Prob>|t| 

<.0001 

0.1763 

0.0723 

0.0178 

0.1134 

0.0654 

0.0264 

0.1308 

0.0469 

0.0035 

0.0003 

0.0381 

0.0466 

0.1238 

<.0001 

0.1945 

0.0001 


Sum  of  Squares 
2.419592 
4.303907 
7.571654 
3.326867 
4.524383 
6.621548 
3.026957 
5.280131 
11.654100 
17.995573 
5.76081 1 
5.291409 
3.142885 
23.715213 
2.226030 
20.468789 


F  Ratio  Prob  >  F 

1.8535  0.1763 

3.2969  0.0723 

5.8001  0.0178 

2.5485  0.1134 

3.4658  0.0654 

5.0723  0.0264 

2.3187  0.1308 

4.0448  0.0469 

8.9274  0.0035 

13.7852  0.0003 

4.4130  0.0381 

4.0534  0.0466 

2.4076  0.1238 

18.1666  <.0001 

1.7052  0.1945 

15.6798  0.0001 


TurfWar  =  4.62  +  sys-embed(0.47)  +  comp-CASE(-1 .04)  +  comp-OS(-1 .07)  +  comp-dev(0.48)  +  ent-acct(0.80)  +  ent- 
mnft(1.42)  +  ent-web(0.63)  +  proc-req(0.54)  +  proc-reeng(-0.97)  +  proc-appsup(-1.15)  +  proc-train(-0.68)  +  proc-coding(- 
0.60)  +  proc-CM(-0.51)  +  proc-toolsup(1.55)  +  proc-none(-1.12)  +  prod-COTS(1.16) 
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Stepwise  Fit  -  Combined  Survey  Data  -  Consequences  (FailLikely) 

Response: 

Column  59 


Stepwise  Regression  Control 

Prob  to  Enter  0.250 

Prob  to  Leave  0.250 

Direction: 

Rules: 


Current  Estimates 

SSE  DFE 

166.39835  109 


MSE  RSquare 

1.5265904  0.3228 


RSquare  Adj 
0.2358 


Cp  AIC 

1.7688794  66.46878 


Lock 

X 


X  Intercept 

whattypesystemsavionics{0-1} 
whattypesystemsembedded{0-1 } 
whattypesystemscommunications{0-1} 
whattypesystemsdevice{0-1} 
whattypeshrinkbusiness{0-1} 
whattypeshrinkutilities{0-1 } 

X  whattypeshrinkinternet{0-1} 

X  whattypecomponentdomain{0-1} 

X  whattypecomponentCASE{0-1} 

X  whattypecomponentclass{0-1} 

X  whattypecomponentOS{0-1} 

X  whattypecomponentdevelopment{0-1} 

whattypeenterpriseacctng{0-1 } 

X  whattypeenterprisemanufact{0-1} 

X  whattypeenterprisepayroll{0-1} 

whattypeenterpriseOES{0- 1 } 
whattypeenterprisescripting{0-1} 
whattypeenterpriseweb{0-1 } 
whatprocessrequirements{0-1} 

X  whatprocessdesign{0-1} 

whatprocesstesting{0-1} 
whatprocessmaintenance{0-1} 
whatprocessreengineering{0-1 } 
whatprocessappsuppt{0-1 } 

X  whatprocesstraining{0-1} 

whatprocessspecification{0-1 } 
whatprocessdocumentation{0-1 } 
whatprocesscoding{0- 1 } 
whatprocessfielding{0-1} 

X  whatprocessCM{0-1} 

X  whatprocesstoolsuppt{0-1} 

whatprocessSWEngSuppt{0- 1 } 
whatprocessnone{0-1 } 

X  whatproductscustom{0-1} 

whatproductsCOTS{0-1} 

X  whatproductscommoncust{0-1} 

whatproductsnone{0-1 } 


Step  History 

Step  Parameter 

1  whattypecomponentclass{0-1} 

2  whatproductscommoncust{0-1} 

3  whatproductscustom{0-1} 

4  whattypeenterprisemanufact{0-1} 

5  whatprocessCM{0-1} 

6  whatprocesstoolsuppt{0-1} 

7  whattypecomponentCASE{0-1} 

8  whatprocessdesign{0-1} 

9  whattypeenterprisepayroll{0-1} 

1 0  whattypecomponentdevelopment{0- 1 } 

1 1  whattypecomponentdomain{0-1 } 

12  whatprocesstraining{0-1} 

13  whattypeshrinkinternet{0-1} 

14  whattypecomponentOS{0-1} 


Action 

Entered 

Entered 

Entered 

Entered 

Entered 

Entered 

Entered 

Entered 

Entered 

Entered 

Entered 

Entered 

Entered 

Entered 


Estimate  nDF 

3.95055205  1 

0  1 

0  1 

0  1 

0  1 

0  1 

0  1 

-0.2578722  1 

-0.2754059  1 

0.45796955  1 

1.22898413  1 

0.29528485  1 

-0.2013401  1 

0  1 

-0.670198  1 

-0.5688313  1 

0  1 

0  1 

0  1 

0  1 

-0.265596  1 

0  1 

0  1 

0  1 

0  1 

0.24963291  1 

0  1 

0  1 

0  1 

0  1 

0.63522587  1 

-0.7348127  1 

0  1 

0  1 

-0.2137006  1 

0  1 

-0.413646  1 

0  1 


SS 

0 

0.370571 
0.017659 
0.001352 
1.213651 
0.001915 
1.17976 
2.810802 
2.149182 
3.496079 
18.47525 
2.46399 
2.755115 
0.531129 
6.388775 
4.94407 
0.592608 
1 .305427 
0.000002 
0.894859 
5.376917 
0.194076 
1.555584 
0.461737 
0.17235 
3.709239 
0.568556 
0.881279 
1.188867 
0.762618 
19.60305 
24.08974 
0.491541 
0.179422 
3.989618 

I. 578858 

II. 99162 
0.119167 


"F  Ratio" 
0.000 
0.241 
0.011 
0.001 
0.794 
0.001 
0.771 
1.841 
1.408 
2.290 
12.102 
1.614 
1.805 
0.346 
4.185 
3.239 
0.386 
0.854 
0.000 
0.584 
3.522 
0.126 
1.019 
0.301 
0.112 
2.430 
0.370 
0.575 
0.777 
0.497 
12.841 
15.780 
0.320 
0.117 
2.613 
1.035 
7.855 
0.077 


'Prob>F" 

1.0000 

0.6244 

0.9149 

0.9764 

0.3750 

0.9719 

0.3818 

0.1776 

0.2380 

0.1331 

0.0007 

0.2066 

0.1819 

0.5577 

0.0432 

0.0747 

0.5357 

0.3575 

0.9992 

0.4464 

0.0632 

0.7232 

0.3150 

0.5847 

0.7386 

0.1220 

0.5441 

0.4499 

0.3800 

0.4822 

0.0005 

0.0001 

0.5728 

0.7334 

0.1089 

0.3114 

0.0060 

0.7814 


'Sig  Prob"  Seq  SS 

0.0187  10.92985 

0.0076  13.499 

0.0700  5.994467 

0.0673  5.998143 

0.0975  4.833965 

0.0021  16.00938 

0.1551  3.269241 

0.1635  3.112914 

0.1298  3.643562 

0.2121  2.451961 

0.2167  2.392106 

0.2082  2.470708 

0.2294  2.242039 

0.2066  2.46399 


RSquare  Cp  p 

0.0445  15.125  2 

0.0994  9.3559  3 

0.1238  7.9059  4 

0.1482  6.4537  5 

0.1679  5.6716  6 

0.2331  -1.542  7 

0.2464  -1.424  8 

0.2590  -1.216  9 

0.2739  -1.313  10 

0.2838  -0.724  1 1 

0.2936  -0.101  12 

0.3036  0.4774  13 

0.3128  1.187  14 

0.3228  1.7689  15 


Appendix  D 


Response  Column  59 
Actual  by  Predicted  Plot 


Column  59  Predicted  P<.0001  RSq=0.32 
RMSE=1.2356 


Summary  of  Fit 


RSquare 

0.322785 

RSquare  Adj 

0.235803 

Root  Mean  Square  Error 

1.235553 

Mean  of  Response 

4.048387 

Observations  (or  Sum  Wgts) 

Analysis  of  Variance 

124 

Source  DF 

Sum  of  Squares  Mean  Square 

F  Ratio 

Model  14 

79.31132 

5.66509 

3.7109 

Error  1 09 

166.39835 

1 .52659 

Prob  >  F 

C.  Total  123 

245.70968 

<.0001 

Lack  Of  Fit 

Source  DF 

Sum  of  Squares 

Mean  Square 

F  Ratio 

Lack  Of  Fit  50 

80.62057 

1.61241 

1.1091 

Pure  Error  59 

85.77778 

1.45386 

Prob  >  F 

Total  Error  109 

166.39835 

0.3494 

Max  RSq 

0.6509 

Parameter  Estimates 

Term 

Estimate 

Std  Error 

t  Ratio  Prob>|t| 

Intercept 

3.2162465 

0.279705 

11.50  <.0001 

whattypeshrinkinternet[1  -0] 

0.5157443 

0.380085 

1.36  0.1776 

whattypecomponentdomain[1-0] 

0.5508117 

0.464224 

1.19  0.2380 

whattypecomponentCASE[1  -0] 

-0.915939 

0.605253 

-1.51  0.1331 

whattypecomponentclass[1  -0] 

-2.457968 

0.706549 

-3.48  0.0007 

whattypecomponentOS[1  -0] 

-0.59057 

0.46485 

-1.27  0.2066 

whattypecomponentdevelopment[1-0] 

0.4026802 

0.299745 

1.34  0.1819 

whattypeenterprisemanufact[1  -0] 

1.3403961 

0.655218 

2.05  0.0432 

whattypeenterprisepayrolljl  -0] 

1.1376626 

0.632168 

1.80  0.0747 

wh  atprocessdesig  n  [  1  -0] 

0.5311919 

0.283039 

1.88  0.0632 

whatprocesstraining[1  -0] 

-0.499266 

0.320295 

-1.56  0.1220 

whatprocessCM[1  -0] 

-1.270452 

0.354534 

-3.58  0.0005 

whatprocesstoolsuppt[1  -0] 

1.4696255 

0.369958 

3.97  0.0001 

whatproductscustom[1  -0] 

0.4274012 

0.264382 

1.62  0.1089 

whatproductscommoncust[1  -0] 

Effect  Tests 

0.8272921 

0.295176 

2.80  0.0060 

Source 

Nparm 

DF 

Sum  of  Squares 

F  Ratio 

Prob  >  F 

whattypeshrinkinternet 

1 

1 

2.810802 

1.8412 

0.1776 

whattypecomponentdomain 

1 

1 

2.149182 

1 .4078 

0.2380 

whattypecomponentCASE 

1 

1 

3.496079 

2.2901 

0.1331 

whattypecomponentclass 

1 

1 

18.475254 

12.1023 

0.0007 

whattypecomponentOS 

1 

1 

2.463990 

1.6140 

0.2066 

whattypecomponentdevelopment 

1 

1 

2.755115 

1.8048 

0.1819 

whattypeenterprisemanufact 

1 

1 

6.388775 

4.1850 

0.0432 

whattypeenterprisepayroll 

1 

1 

4.944070 

3.2386 

0.0747 

whatprocessdesign 

1 

1 

5.376917 

3.5222 

0.0632 

wh  atprocesstrai  n  i  ng 

1 

1 

3.709239 

2.4298 

0.1220 

whatprocessCM 

1 

1 

19.603051 

12.8411 

0.0005 

whatprocesstoolsuppt 

1 

1 

24.089740 

15.7801 

0.0001 

whatproductscustom 

1 

1 

3.989618 

2.6134 

0.1089 

whatproductscommoncust 

1 

1 

11.991615 

7.8552 

0.0060 

FailLikely  =  3.22  +  shrink-int(0.52)  +  comp-domain(0.55)  +  comp-CASE(-0.92)  +  comp-class(-2.46)  +  comp-OS(-0.59)  + 
comp-dev(0.40)  +  ent-mnft(1.34)  +  ent-pay(1.14)  +  proc-des(0.53)  +  proc-train(-0.50)  +  proc-CM(-1.27)  +  proc- 
toolsup(1.47)  +  prod-cust(0.43)  +  prod-comcust(0.83) 
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Stepwise  Fit  -  Combined  Survey  Data  -  Consequences  (RespCust) 

Response: 

Column  59 


Stepwise  Regression  Control 

Prob  to  Enter  0.250 

Prob  to  Leave  0.250 

Direction: 

Rules: 


Current  Estimates 

SSE  DFE 

157.22697  108 


MSE  RSquare 

1.4558052  0.2738 


RSquare  Adj 
0.1662 


Cp  AIC 

2.0078492  62.67208 


Lock 

X 


X  Intercept 

whattypesystemsavionics{0-1} 

X  whattypesystemsembedded{0-1} 

whattypesystemscommunications{0-1} 

whattypesystemsdevice{0-1} 

whattypeshrinkbusiness{0-1} 

X  whattypeshrinkutilities{0-1} 

whattypeshrinkinternet{0-1 } 

X  whattypecomponentdomain{0-1} 

X  whattypecomponentCASE{0-1} 

whattypecomponentclass{0- 1 } 
whattypecomponentOS{0-1 } 
whattypecomponentdevelopment{0-1} 
whattypeenterpriseacctng{0-1 } 
whattypeenterprisemanufact{0-1 } 
whattypeenterprisepayroll{0-1 } 

X  whattypeenterpriseOES{0-1} 

whattypeenterprisescripting{0-1} 

X  whattypeenterpriseweb{0-1} 

X  whatprocessrequirements{0-1} 

whatprocessdesign{0- 1 } 
whatprocesstesting{0-1} 
whatprocessmaintenance{0-1} 
whatprocessreengineering{0-1} 
whatprocessappsuppt{0-1 } 

X  whatprocesstraining{0-1} 

X  whatprocessspecification{0-1} 

whatprocessdocumentation{0-1 } 
whatprocesscoding{0- 1 } 

X  whatprocessfielding{0-1} 

X  whatprocessCM{0-1} 

whatprocesstoolsuppt{0-1} 

X  whatprocessSWEngSuppt{0-1} 

X  whatprocessnone{0-1} 

X  whatproductscustom{0-1} 

X  whatproductsCOTS{0-1} 

X  whatproductscommoncust{0-1} 

whatproductsnone{0-1 } 


Step  History 

Step  Parameter 

1  whatprocesstraining{0-1} 

2  whattypecomponentCASE{0-1} 

3  whatprocessfielding{0-1} 

4  whatprocessCM{0-1} 

5  whatprocessSWEngSuppt{0-1} 

6  whatprocessspecification{0-1} 

7  whatprocessrequirements{0-1} 

8  whattypeshrinkutilities{0-1} 

9  whattypeenterpriseOES{0-1} 

10  whatproductsCOTS{0-1} 

1 1  whatproductscustom{0-1 } 

12  whatproductscommoncust{0-1} 

13  whatprocessnone{0-1} 

14  whattypesystemsembedded{0-1} 

15  whattypecomponentdomain{0-1} 

16  whattypeenterpriseweb{0-1} 


Action 

Entered 

Entered 

Entered 

Entered 

Entered 

Entered 

Entered 

Entered 

Entered 

Entered 

Entered 

Entered 

Entered 

Entered 

Entered 

Entered 


Estimate  nDF 

4.61847758  1 

0  1 

0.25344136  1 

0  1 

0  1 

0  1 

-0.6894193  1 

0  1 

0.31037685  1 

-0.7770687  1 

0  1 

0  1 

0  1 

0  1 

0  1 

0  1 

-0.5211476  1 

0  1 

0.22597924  1 

-0.3574829  1 

0  1 

0  1 

0  1 

0  1 

0  1 

-0.3401992  1 

0.37085993  1 

0  1 

0  1 

0.41361452  1 

-0.4635919  1 

0  1 

0.39637726  1 

0.70355886  1 

0.23379276  1 

0.33350658  1 

0.19604238  1 

0  1 


"Sig  Prob" 
0.1009 
0.0997 
0.0822 
0.1026 
0.0291 
0.1350 
0.1183 
0.1063 
0.1219 
0.1910 
0.1805 
0.1910 
0.1555 
0.2389 
0.2286 
0.2186 


Seq  SS 
4.705069 
4.673597 
5.126642 
4.453593 
7.781651 
3.574227 
3.857667 
4.071727 
3.686133 
2.610475 
2.720252 
2.5751 1 
3.015157 
2.060051 
2.144318 
2.229366 


SS 

0 

0.245073 

2.852401 

0.928216 

0.07507 

0.082611 

7.407935 

1.299606 

2.385032 

9.889576 

0.484837 

0.054864 

0.020271 

0.004567 

1.351214 

0.000259 

2.025051 

0.178287 

2.229366 

7.439745 

0.024247 

1.117871 

0.115248 

0.31358 

I. 104345 
6.508563 
8.876244 
0.294804 

0.3672 

7.096394 

I I . 66933 
0.402726 
9.736792 
3.612233 
4.062698 
6.245348 
2.677357 

0.23399 


RSquare 

0.0217 

0.0433 

0.0670 

0.0876 

0.1235 

0.1400 

0.1578 

0.1766 

0.1937 

0.2057 

0.2183 

0.2302 

0.2441 

0.2536 

0.2635 

0.2738 


"F  Ratio"  "Prob>F" 

0.000  1.0000 

0.167  0.6836 

1.959  0.1645 

0.635  0.4271 

0.051  0.8216 

0.056  0.8130 

5.089  0.0261 

0.892  0.3471 

1.638  0.2033 

6.793  0.0104 

0.331  0.5663 

0.037  0.8471 

0.014  0.9067 

0.003  0.9556 

0.928  0.3377 

0.000  0.9894 

1.391  0.2408 

0.121  0.7281 

1.531  0.2186 

5.110  0.0258 

0.017  0.8980 

0.766  0.3834 

0.078  0.7799 

0.214  0.6447 

0.757  0.3863 

4.471  0.0368 

6.097  0.0151 

0.201  0.6548 

0.250  0.6178 

4.875  0.0294 

8.016  0.0055 

0.275  0.6012 

6.688  0.0110 

2.481  0.1181 

2.791  0.0977 

4.290  0.0407 

1.839  0.1779 

0.159  0.6904 


Cp  p 

4.2947  2 

3.53  3 

2.4974  4 

1.8628  5 

-0.74  6 

-0.855  7 

-1.137  8 

-1.545  9 

-1.726  10 

-1.27  11 

-0.879  12 

-0.403  13 

-0.186  14 

0.5951  15 

1.3266  16 

2.0078  17 
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Response  Column  59 
Actual  by  Predicted  Plot 


Column  59  Predicted  P=0.0023  RSq=0.27 
RMSE=1 .2066 


Summary  of  Fit 


RSquare  0.273819 

RSquare  Adj  0.166236 

Root  Mean  Square  Error  1 .206568 

Mean  of  Response  4.352 

Observations  (or  Sum  Wgts)  125 


Analysis  of  Variance 


Source  DF 

Model  16 

Error  1 08 

C.  Total  124 


Sum  of  Squares 
59.28503 
157.22697 
216.51200 


Mean  Square  F  Ratio 

3.70531  2.5452 

1 .45581  Prob  >  F 

0.0023 


Parameter  Estimates 

Term 

Intercept 

whattypesystemsembedded[1  -0] 
whattypeshrinkutilities[1  -0] 
whattypecomponentdomain[1  -0] 
whattypecomponentCASE[1  -0] 
whattypeenterpriseOES[1  -0] 
whattypeenterpriseweb[1  -0] 
whatprocessrequirements[1  -0] 
whatprocesstraining[1-0] 
whatprocessspecification[1  -0] 
whatprocessfielding[1-0] 
whatprocessCMjl  -0] 
whatprocessSWEngSuppt[1  -0] 
whatprocessnone[1  -0] 
whatproductscustom[1-0] 
whatproductsCOTS[1-0] 
whatproductscommoncust[1-0] 


Estimate 

4.9071177 

-0.506883 

1.3788387 

-0.620754 

1.5541374 

1.0422951 

-0.451958 

0.7149658 

0.6803984 

-0.74172 

-0.827229 

0.9271838 

-0.792755 

-1.407118 

-0.467586 

-0.667013 

-0.392085 


Std  Error 
0.264709 
0.362121 
0.611247 
0.48498 
0.596283 
0.88374 
0.365224 
0.31627 
0.32179 
0.300384 
0.374678 
0.327487 
0.306537 
0.893294 
0.279902 
0.322038 
0.28912 


t  Ratio 
18.54 
-1.40 
2.26 
-1.28 
2.61 
1.18 
-1.24 
2.26 
2.11 
-2.47 
-2.21 
2.83 
-2.59 
-1.58 
-1.67 
-2.07 
-1.36 


Prob>|t| 

<.0001 

0.1645 

0.0261 

0.2033 

0.0104 

0.2408 

0.2186 

0.0258 

0.0368 

0.0151 

0.0294 

0.0055 

0.0110 

0.1181 

0.0977 

0.0407 

0.1779 


Effect  Tests 

Source 

whattypesystemsembedded 

whattypeshrinkutilities 

whattypecomponentdomain 

whattypecomponentCASE 

whattypeenterpriseOES 

whattypeenterpriseweb 

whatprocessrequirements 

whatprocesstraining 

whatprocessspecification 

whatprocessfielding 

whatprocessCM 

whatprocessSWEngSuppt 

whatprocessnone 

whatproductscustom 

whatproductsCOTS 

whatproductscommoncust 


Nparm  DF 

1  1 

1  1 

1  1 

1  1 

1  1 

1  1 

1  1 

1  1 

1  1 

1  1 

1  1 

1  1 

1  1 

1  1 

1  1 

1  1 


Sum  of  Squares 
2.852401 
7.407935 
2.385032 
9.889576 
2.025051 
2.229366 
7.439745 
6.508563 
8.876244 
7.096394 
11.669333 
9.736792 
3.612233 
4.062698 
6.245348 
2.677357 


F  Ratio  Prob  >  F 

1.9593  0.1645 

5.0885  0.0261 

1 .6383  0.2033 

6.7932  0.0104 

1.3910  0.2408 

1.5314  0.2186 

5.1104  0.0258 

4.4708  0.0368 

6.0971  0.0151 

4.8745  0.0294 

8.0157  0.0055 

6.6883  0.0110 

2.4813  0.1181 

2.7907  0.0977 

4.2900  0.0407 

1.8391  0.1779 


ResponseCustomer  =  4.91  +  sys-embed(-0.51)  +  shrink-util(1 .38)  +  comp-domain(-0.62)  +  comp-CASE(1.55)  +  ent- 
OES(1.04)  +  ent-web(-0.45)  +  proc-req(0.71)  +  proc-train(0.68)  +  proc-spec(-0.74)  +  proc-field(-0.83)  +  proc-CM(0.93)  + 
proc-SWEngSup(-0.79)  +  proc-none(-1.41)  +  prod-cust(-0.47)  +  prod-COTS(-0.67)  +  prod-comcust(-0.39) 
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Stepwise  Fit  -  Combined  Survey  Data  -  Consequences  (ResponseOrg) 

Response: 

Column  59 


Stepwise  Regression  Control 

Prob  to  Enter  0.250 

Prob  to  Leave  0.250 

Direction: 

Rules: 


Current  Estimates 

SSE  DFE 

214.07154  113 


MSE  RSquare 

1.8944384  0.1988 


RSquare  Adj 
0.1208 


Cp  AIC 

-5.89511  91.24956 


Lock 

X 


X  Intercept 

whattypesystemsavionics{0-1} 
whattypesystemsembedded{0-1 } 

X  whattypesystemscommunications{0-1 } 

X  whattypesystemsdevice{0-1} 

whattypeshrinkbusiness{0-1} 
whattypeshrinkutilities{0-1 } 
whattypeshrinkinternet{0-1 } 
whattypecomponentdomain{0-1} 

X  whattypecomponentCASE{0-1} 

whattypecomponentclass{0- 1 } 

X  whattypecomponentOS{0-1} 

whattypecomponentdevelopment{0-1} 
whattypeenterpriseacctng{0-1 } 

X  whattypeenterprisemanufact{0-1} 

whattypeenterprisepayroll{0-1 } 

X  whattypeenterpriseOES{0-1} 

whattypeenterprisescripting{0-1} 
whattypeenterpriseweb{0-1 } 

X  whatprocessrequirements{0-1} 

whatprocessdesign{0- 1 } 
whatprocesstesting{0- 1 } 
whatprocessmaintenance{0-1} 
whatprocessreengineering{0-1} 
whatprocessappsuppt{0-1 } 
whatprocesstraining{0-1} 
whatprocessspecification{0-1 } 
whatprocessdocumentation{0-1 } 

X  whatprocesscoding{0-1} 

X  whatprocessfielding{0-1} 

X  whatprocessCM{0-1} 

whatprocesstoolsuppt{0-1} 
whatprocessSWEngSuppt{0- 1 } 
whatprocessnone{0-1 } 

X  whatproductscustom{0-1} 

whatproductsCOTS{0-1} 
whatproductscommoncust{0-1} 
whatproductsnone{0-1 } 


Step  History 

Step  Parameter 

1  whatproductscustom{0-1} 

2  whattypeenterprisemanufact{0-1} 

3  whattypecomponentCASE{0-1} 

4  whattypesystemsdevice{0-1} 

5  whattypecomponentOS{0-1} 

6  whatprocessCM{0-1) 

7  whatprocessfielding{0-1} 

8  whattypeenterpriseOES{0-1} 

9  whattypesystemscommunications{0-1} 

10  whatprocessrequirements{0-1} 

11  whatprocesscoding{0-1} 


Estimate  nDF 

4.94532414  1 

0  1 

0  1 

-0.2688162  1 

0.47834009  1 

0  1 

0  1 

0  1 

0  1 

-0.8414831  1 

0  1 

-0.3677225  1 

0  1 

0  1 

0.64825314  1 

0  1 

-0.7031229  1 

0  1 

0  1 

0.25732566  1 

0  1 

0  1 

0  1 

0  1 

0  1 

0  1 

0  1 

0  1 

-0.2029815  1 

0.46621156  1 

-0.4565346  1 

0  1 

0  1 

0  1 

0.25523252  1 

0  1 

0  1 

0  1 


SS 

0 

0.000022 
0.558277 
5.275968 
6.976999 
0.666903 
0.027499 
0.024793 
1 .03364 
11.93592 
0.198188 
3.756402 
0.361773 
0.052845 
6.105685 
0.0287 
3.767142 
1 .776085 
0.0025 
5.270874 
2.073217 
0.006896 
1.331687 
0.610359 
0.006254 
1.216748 
1.135903 
0.489486 
2.69195 
9.50673 
12.52597 
0.063618 
0.450038 
0.190592 
6.263891 
1.448571 
0.05577 
0.013771 


"F  Ratio"  "Prob>F" 

0.000  1.0000 

0.000  0.9973 

0.293  0.5895 

2.785  0.0979 

3.683  0.0575 

0.350  0.5553 

0.014  0.9047 

0.013  0.9095 

0.543  0.4626 

6.301  0.0135 

0.104  0.7479 

1.983  0.1618 

0.190  0.6641 

0.028  0.8682 

3.223  0.0753 

0.015  0.9027 

1.989  0.1612 

0.937  0.3351 

0.001  0.9712 

2.782  0.0981 

1.095  0.2976 

0.004  0.9522 

0.701  0.4042 

0.320  0.5726 

0.003  0.9545 

0.640  0.4253 

0.597  0.4412 

0.257  0.6134 

1.421  0.2357 

5.018  0.0270 

6.612  0.0114 

0.033  0.8555 

0.236  0.6281 

0.100  0.7527 

3.306  0.0717 

0.763  0.3842 

0.029  0.8647 

0.007  0.9325 


Action 

Entered 

Entered 

Entered 

Entered 

Entered 

Entered 

Entered 

Entered 

Entered 

Entered 

Entered 


"Sig  Prob"  Seq  SS 

0.0947  6.023529 

0.1195  5.159982 

0.0844  6.251471 

0.0990  5.624373 

0.1303  4.670796 

0.1404  4.388318 

0.0554  7.291518 

0.2018  3.190133 

0.1937  3.288992 

0.1248  4.547398 

0.2357  2.69195 


RSquare  Cp  p 

0.0225  -4.968  2 

0.0419  -5.26  3 

0.0653  -6.038  4 

0.0863  -6.536  5 

0.1038  -6.611  6 

0.1202  -6.561  7 

0.1475  -7.8  8 

0.1594  -7.218  9 

0.1717  -6.679  10 

0.1888  -6.699  11 

0.1988  -5.895  12 
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Response  Column  59 
Actual  by  Predicted  Plot 


i  |  i  |  i  |  i  |  i  |  i 

1  2  3  4  5  6  7 

Column  59  Predicted  P=0.0064  RSq=0.20 
RMSE=1.3764 


Summary  of  Fit 


RSquare 
RSquare  Adj 
Root  Mean  Square  Error 
Mean  of  Response 
Observations  (or  Sum  Wgts) 
Analysis  of  Variance 


Source  DF 

Model  11 

Error  113 

C.  Total  124 


0.198834 

0.120844 

1.376386 

4.28 

125 

Sum  of  Squares 
53.12846 
214.07154 
267.20000 


Mean  Square  F  Ratio 

4.82986  2.5495 

1 .89444  Prob  >  F 

0.0064 


Lack  Of  Fit 


Source  DF 

Lack  Of  Fit  36 

Pure  Error  77 

Total  Error  113 


Sum  of  Squares  Mean  Square 

80.28582  2.23016 

133.78571  1.73748 

214.07154 


F  Ratio 
1.2836 
Prob  >  F 
0.1795 
Max  RSq 
0.4993 


Parameter  Estimates 

Term 

Intercept 

whattypesystemscommunications[1-0] 
whattypesystemsdevice[1  -0] 
whattypecomponentCASE[1  -0] 
whattypecomponentOS[1  -0] 
whattypeenterprisemanufact[1  -0] 
whattypeenterpriseOES[1  -0] 
whatprocessrequirements[1  -0] 
whatprocesscoding[1-0] 
whatprocessfielding[1  -0] 
whatprocessCM[1  -0] 
whatproductscustom[1  -0] 


Estimate 
4.2100264 
0.5376324 
-0.95668 
1.6829661 
0.7354449 
-1.296506 
1 .4062458 
-0.514651 
0.4059629 
-0.932423 
0.9130692 
-0.510465 


Std  Error 
0.33382 
0.322162 
0.498509 
0.670483 
0.522281 
0.722184 
0.99723 
0.30854 
0.34056 
0.416234 
0.35509 
0.280727 


t  Ratio 
12.61 
1.67 
-1.92 
2.51 
1.41 
-1.80 
1.41 
-1.67 
1.19 
-2.24 
2.57 
-1.82 


Prob>|t| 

<.0001 

0.0979 

0.0575 

0.0135 

0.1618 

0.0753 

0.1612 

0.0981 

0.2357 

0.0270 

0.0114 

0.0717 


Effect  Tests 

Source 

whattypesystemscommunications 

whattypesystemsdevice 

whattypecomponentCASE 

whattypecomponentOS 

whattypeenterprisemanufact 

whattypeenterpriseOES 

whatprocessrequirements 

whatprocesscoding 

whatprocessfielding 

whatprocessCM 

whatproductscustom 


Nparm  DF 

1  1 

1  1 

1  1 

1  1 

1  1 

1  1 

1  1 

1  1 

1  1 

1  1 

1  1 


Sum  of  Squares 
5.275968 
6.976999 
11.935924 
3.756402 
6.105685 
3.767142 
5.270874 
2.691950 
9.506730 
12.525974 
6.263891 


F  Ratio  Prob  >  F 

2.7850  0.0979 

3.6829  0.0575 

6.3005  0.0135 

1.9829  0.1618 

3.2230  0.0753 

1.9885  0.1612 

2.7823  0.0981 

1.4210  0.2357 

5.0182  0.0270 

6.6120  0.0114 

3.3065  0.0717 


ResponseOrg  =  4.21  +  sys-comm(0.54)  +  sys-dev(-0.96)  +  comp-CASE(1.68)  +  comp-OS(0.74)  +  ent-mnft(-1 .30)  +  ent- 
OES(1.41)  +  proc-req(-0.51)  +  proc-coding(0.41)  +  proc-field(-0.93)  +  proc-CM(0.91)  +  prod-cust(-0.51) 
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Appendix  E  -  Cross  Validation  Neurosolution ’s  Screen  Shots 
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Appendix  F  -  Training  Neurosolution ’s  Screen  Shots 
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Software  Outsourcing  —  Study  Objectives 


We  are  researchers  in  the  Arizona  State  University's  Computer  Science  and  Engineering  Department  wno  are 
investigating  software  development  outsourcing.  According  to  published  accounts,  software  development 
outsourcing  has  become  commonplace  and  often  meets 
organizational  goals.  Unfortunately,  nearly  30%  of  outsourcing 
relationships  end  poorly  (anything  from  general  dissatisfaction  to 
legal  action). 

With  your  assistance,  we  hope  to  identify  software  outsourcing 
strategies,  motivations,  benefits,  drawbacks,  and  relevant  project 
situation  variables.  This  information  will  help  us  to  discern  why 
outsourcing  efforts  succeed  or  fail  to  meet  goals  and  which  strategies 
are  most  appropriate  for  specific  projects  and  goals.  Using  this  knowledge,  we  will  produce  a  process  simulation 
tool  which  will  allow  researchers  and  project  managers  to  more  closely  study  the  inter-organizational  relations 
within  a  planned  outsourcing  relationship  and  their  impact  on  the  overall  software  development  process.  A  second 
tool,  for  decision  support,  will  then  be  constructed  to  aid  software  development  project  managers  and  consultants  in 
making  software  outsourcing  strategy  decisions  for  specific  projects. 

Who  can  help? 

You  can  help  by  completing  this  brief  survey  if,  within  the  last  2  years,  you  have  participated  in  a  software 
development  project  where  any  portion  of  the  product  development  or  effort  has  been  contracted  to  an  outside 
vendor  (regardless  of  which  side  of  the  relationship  you  worked  on).  This  survey  includes  questions  about  your 
background,  your  most  recent  software  outsourcing  project,  and  general  outsourcing  experience  over  the  past  five 
years.  The  questionnaire  is  designed  to  take  less  than  15  minutes  to  complete. 

What  do  I  get  for  helping? 

If  you  choose  to  participate,  your  answers  will  be  held  in  the  strictest  confidence.  Only  our  research  team  will  see 
your  individual  answers.  Our  reports  will  consist  of  summaries  of  data  from  all  respondents.  When  completed 
(planned  for  late  spring  1999),  these  summary  reports  will  be  available  to  survey  participants  via  our  outsourcing 
website  (http://www.eas.asu.edu/~outsrc/).  If  you  provide  the  optional  contact  information,  you  will  be  notified 
when  survey  results  are  posted  and  will  be  provided  with  free  copies  of  the  decision  support  tools  when  they 
become  available. 


Definition 

Software  development  outsourcing: 

hiring  of  vendors  to  perform  software 
development  activities  or  develop  a 
portion  of  an  overall  software  product. 
It  does  not  include  the  hiring  of 
temporary  employees. 


Feel  free  to  contact  us  with  any  questions  you  might  have  regarding  our  research.  Thank  you  for  your  assistance. 


Brian  G.  Hermann 

Ph.D.  Candidate 

Department  of  Computer  Science  and  Engineering 
Arizona  State  University 
brian.hermann@asu.edu 


Stephen  T.  Roehling 

Master's  Candidate 

Department  of  Computer  Science  and  Engineering 
Arizona  State  University 
roehling@imap3.asu.edu 


(OutSurveyF  inal) 
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Instructions 


I. 


1.  For  the  purposes  of  this  study,  we  define  software  development  outsourcing  as  the  hiring  of  vendors  to  perform 
software  development  activities  or  to  develop  a  portion  of  a  software  product.  It  does  not  include  the  hiring  of 
temporary  employees. 

2.  Please  answer  every  question.  Some  questions  may  look  like  others,  but  each  one  is  different. 

3.  There  are  no  right  or  wrong  answers.  Please  provide  a  realistic  assessment  of  each  item  based  on  your 
experiences.  The  focus  of  the  survey  is  on  your  experience,  not  on  what  you  wish  were  true  or  what  may  be  true 
in  the  future. 

4.  For  questions  pertaining  to  this  survey  please  contact  Brian  Hermann  via  e-mail  at  brian.hermann@asu.edu. 

5.  Please  return  this  survey  to: 

Brian  G.  Hermann 

Computer  Science  and  Engineering  Department 
College  of  Engineering  and  Applied  Sciences 
Arizona  State  University 
Box  875406 

Tempe,  AZ  85287-5406,  U.S.A. 

6.  Please  remove  this  page  for  your  information  and  continue  with  the  survey. 


Confidentiality 


Your  responses  to  this  survey  are  confidential.  As  summarized  below,  no  organization  or  individual 
respondent  will  be  identified  by  name  in  any  analyses  or  report  without  your  written  permission. 


PUBLIC  LAW  93-759,  entitled  the  Privacy  Act  of  1974  requires  that  all  individuals  be  informed  of  the  purposes  and 
uses  to  be  made  of  the  information  which  is  solicited.  The  following  is  furnished  to  explain  why  the  information  is 
requested  and  the  general  uses  to  which  the  information  may  be  put. 


Purpose:  This  study  strives  to  examine  software  outsourcing  strategies,  motivations,  benefits,  drawbacks,  and 
relevant  project  situation  variables.  The  survey  results  will  be  used  both  to  better  understand  software  outsourcing, 
as  well  as  to  develop  a  software  outsourcing  process  simulation  tool. 


Uses:  Survey  data  are  used  for  research  purposes  only.  Individual  responses  are  confidential.  Only  summarized  data 
will  be  reported  to  you,  if  you  so  request,  and  academic  audiences. 

Effects  of  Non-Disclosure:  Participation  in  the  study  is  voluntary.  No  penalty  will  be  imposed  for  failure  to 
respond  to  any  particular  question. 
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II.  Background  Information 

1.  How  many  software  development  projects  involving  outsourcing  have  you  participated  in  during  the  past 

five  years?  _ Projects 

2.  Roughly,  what  portion  of  your  organization's  software  development  has  been  outsourced  during  the  past 

five  years?  _ % 

3.  The  final  result  of  this  research  will  be  a  suite  of  software  tools  to  help  practitioners  in  various  roles 
evaluate  software  outsourcing  strategies  and  better  understand  software  outsourcing  dynamics  and 
constraints.  Would  these  types  of  tools  be  helpful  to  you? 

□  Yes  □  No 

Why  or  Why  Not? _ 


4.  In  the  future,  would  you  like  to  be  contacted  to  have  the  opportunity  to  provide  inputs  to  other  software 
outsourcing  research  questionnaires  and  to  receive  copies  of  the  software  outsourcing  decision  support 
and  simulation  tools? 

□  Yes  □  No 


5.  Please  provide  your  name  and  best  method  of  contact  (all  results  will  be  kept  confidential) 


Name 

Electronic  mail 

Telephone 

(  )  —  extension 

Standard  Mail 

III.  Most  Recent  Software  Development  Outsourcing  Project  Experience 

Please  answer  the  following  questions  for  the  most  recent  software  outsourcing  project  that  you  worked  on 
(within  the  last  2  years).  If  you  have  worked  on  multiple  projects  recently,  please  answer  the  questions 
utilizing  the  project  about  which  you  have  the  most  knowledge. 


6.  What  type  of  software  was  developed  in  this  project?  Please  check  application  area  (domain)  and/or 
project  type. 

♦  Systems  software,  e.g.:  ♦  Shrink-wrap  commercial/consumer  software 

□  Avionics  products,  e.g.: 

□  Embedded  controllers  and  firm-ware  Q  Entertainment 

□  Communications  systems  Q  Business  productivity 

□  Device  drivers  □  Utilities 

□  Other:  _  □  Internet 

□  Other: 


♦  Software  component  development,  e.g.: 

□  Domain  frameworks 

□  CASE  tools 

□  Class  libraries 

□  Operating  systems 

□  Development  tools 

□  Other: 


♦  Enterprise  software  development  and 
package  customization,  e.g.: 

□  Accounting  systems 

□  Manufacturing  requirements  planning 

□  Payroll  systems 

□  Order  Entry  System 

□  Scripting  and  extensions  development 

□  Interactive  web-site  development 

□  Other: 
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9.  With  respect  to  in-house  development,  what  project  goals  (motivations  for  outsourcing)  were  part  of  the 
decision  to  outsource  software  development  for  this  project? 

Please  note  there  are  two  parts  to  this  question: 

a.  Estimate  the  importance  of  each  goal  using  the  importance  scale  and  the  blanks  to  the  left  of  each  goal. 


b.  Estimate  the  degree  to  which  these  goals  were  realized  by  the  selected  outsourcing  strategy  —  circle  the 
appropriate  number  on  the  scale  to  the  right  of  each  goal. 


Importance  Scale 

Not  Important 

Somewhat  Important 

Very  Important 

1  2 

3 

4 

5 

Significantly 

Exactly  on 
Target 

Significantly 

Importance  Goals 

Worse  than 

Better  than 

Expectations 

Expectations 

Costs  &  Schedule 

a.  Reduce  project  costs  by  taking  advantage  of 

outsourcing  vendor's  economies  of  scale . 

b.  Reduce  development  schedule  —  a  vendor  can 
complete  the  job  faster  than  our  in-house  team  .... 

c.  Reduce  development  schedule  —  parallel  activities 

from  dividing  the  effort  speeds  up  the  overall 
schedule . 

d.  Cash  flow  from  sale  of  the  outsourced  product's 

distribution  rights  to  the  outsourcing  vendor . 


Personnel 

e.  Acquire  expertise  not  available  within  the  internal 

organization  (e.g.  domain,  language,  tool,  etc.) . 

f.  Add  more  personnel  to  the  project  (necessary  due  to 

an  insufficient  in-house  capacity) . 

g.  Add  more  personnel  to  fill  a  short-term,  part-time  or 


transient  need  for  effort  (e.g.,  only  for  fielding  at  the  12  3 

end  of  the  project) . 

h.  Outsource  ‘non-core’  activities .  j  2  3 

i.  Control  over  outsourced  project  management 

process .  1  2  3 

j.  Improved  response  to  customer  objectives .  12  3 

k.  Improved  response  to  organizational  objectives  and 

strategies .  1  2  3 

l.  Keep  in-house  staffing  levels  more  stable .  j  2  3 

General 

m.  Risk  sharing  or  reduction  of  likelihood  and/or 

consequence  (e.g.,  technical,  cost) .  1  2  3 

n.  Product  quality  improvement .  j  2  3 

Other  (please  list) 


4 

4 

4 

4 

4 

4 

4 

4 


4 

4 


5 

5 

5 

5 

5 

5 

5 

5 


5 

5 


o. 


2  3  4  5 


2  3  4  5 


2  3  4  5 


10.  What  were  the  consequences  of  outsourcing  in  this  project  in  comparison  to  similar  in-house  efforts? 
(Put  the  appropriate  number  from  the  consequence  scale  in  the  blank  next  to  each  factor) 


Decreased 

Decreased 

Decreased 

No 

Increased 

Increased 

Increased 

Dramatically 

Significantly 

Slightly 

Change 

Slightly 

Significantly 

Dramatically 

1 

2 

3 

4 

5 

6 

7 

_  a.  Project  costs 

b.  Development  schedule  (vendor  outsourcing  compared  to  in-house) 

_  c.  Intellectual  capital  (your  organization's  rights  to  the  developed  software  product) 

d.  Scheduling  flexibility  (including  ability  to  respond  to  immediate  needs  such  as  a  late  project 
-  productivity  burst) 

_  e.  Administrative  overhead 

_  f.  Control  over  outsourced  project  management  process 
_  g.  In-house  effort  spent  on  ‘non-core’  activities 
_  h.  In-house  personnel  turnover 

_  i.  Project  learning  curve  (time  required  to  become  productive  on  the  project) 
j.  Development  risks 
_  k.  Product  quality 

l.  Rework 

m.  Visibility  into  software  development  process  (ability  to  ascertain  development  progress,  adherence  to 
-  process  standards,  and  product  quality) 

_  n.  Control  over  final  product 

_  o.  Costs  associated  with  design  or  requirements  changes 
_  p.  Cultural,  location,  and  language  problems 

_  q.  Turf  wars  (e.g.  finger  pointing  between  development  groups  —  either  in-house  or  vendors) 

_  r.  Likelihood  of  a  failed  or  cancelled  project 

_  s.  Response  to  customer  objectives 

_  t.  Response  to  organizational  objectives  and  strategies 

Other  (please  list  any  other  outsourcing  consequences  not  already  shown  —  include  impact  rating  if 
appropriate) 


Page  5 


Page  6 


Appendix  G 


Page  3 


11.  On  this  project  who  (or  what  project  roles)  drove  outsourcing  decision  making?  (Select  all  that  apply) 


Outsourcing  Customer  (organization  which 
hires  and  outside  vendor  to  develop  software) 

Outsourcing  Vendor  (organization  which 
develops  software  for  another  organization) 

□  Project  manager 

CJ  Project  manager 

□  Contract  officer 

CJ  Contract  officer 

□  Technical  lead 

D  Technical  lead 

□  Software  developer 

□  Software  developer 

□  Corporate  management  policy 

D  Other  (Please  Explain) 

□  Corporate  management  (one-time  decision) 

D  Management  consultant  working  for  an 

outsourcing  customer 

□  Other  (Please  Explain) 

12.  What  role(s)  did  you  play  in  this  software  outsourcing  relationship?(Select  all  that  apply) 


Outsourcing  Customer  (organization  which 
hires  and  outside  vendor  to  develop  software) 

Outsourcing  Vendor  (organization  which 
develops  software  for  another  organization) 

□  Project  manager 

[J  Project  manager 

□  Contract  officer 

[J  Contract  officer 

D  Technical  lead 

Q  Technical  lead 

□  Software  developer 

□  Software  developer 

□  Management  consultant  working  for  an 
outsourcing  customer 

Q  Other  (Please  Explain) 

[J  Other  (Please  Explain) 

IV.  General  Outsourcing  Experience 

Instructions  -  Consider  outsourcing  projects  you've  worked  on  in  the  last  five  years. 

13.  Based  upon  your  experience,  identify  your  level  of  agreement  with  the  following  assertions  about 
software  development  outsourcing. 

(Put  the  appropriate  number  from  the  scale  in  the  blank  next  to  each  assertion) 

Agreement  Scale 


Strongly  Disagree 

1  2 

Neither  Agree  Nor  Disagree 

3 

4 

Strongly  Agree 

5 

Project  Assertions 

a.  Outsourcing  portions  of  larger  software  development  projects  is  more  successful  than  outsourcing 
portions  of  smaller  software  development  projects. 


b.  Larger  outsourcing  efforts  are  more  successful  than  smaller  outsourcing  efforts. 

c.  Outsourcing  development  of  software  in  some  domains  is  more  successful  than  outsourcing 
development  of  software  in  other  domains. 

d.  Outsourcing  development  of  software  in  a  domain  familiar  to  the  buyer  (in-house  organization)  is 
more  successful  than  outsourcing  development  of  software  in  an  unfamiliar  domain. 

e.  Outsourcing  development  of  software  in  a  domain  familiar  to  the  vendor  is  more  successful  than 
outsourcing  development  of  software  in  a  domain  with  which  the  vendor  is  unfamiliar. 

f.  Outsourcing  development  of  software  is  more  successful  when  more  vendors  are  available  in  the 
project  domain. 

g.  Outsourcing  development  of  software  is  more  successful  when  the  software  vendor  has  more 
experience  with  tools  or  languages. 

h.  Outsourcing  development  of  software  is  more  successful  when  the  software  vendor  has  reusable 
design  or  code  components. 

Buyer-Seller  Relationship  and  Contract  Assertions 

i.  Outsourcing  projects  with  frequent  reviews  and  inspections  are  more  successful  than  outsourcing 
projects  with  less  frequent  reviews  and  inspections. 

j.  Outsourcing  project  success  is  closely  related  to  payment  strategies  and  incentives  in  the  vendor 
contract  (e.g.,  fixed-price  contracts  projects  are  more  or  less  successful  than  cost-plus  type  contracts). 

k.  Outsourcing  project  success  is  closely  tied  to  the  form  of  communication  between  the  buyer  and 
vendor  (forms  of  communication  include  formal  letters,  e-mail,  telephone  conversations,  face-to-face 
meetings,  etc.). 

l.  Outsourcing  projects  are  more  successful  when  the  buyer  has  more  visibility  into  the  vendor's 
development  process. 

m.  Outsourcing  projects  are  more  successful  when  the  buyer  and  vendor  are  located  nearby. 

n.  Outsourcing  projects  are  more  successful  when  the  buyer  and  vendor  are  located  far  apart  (such  as 
"off-shore"  arrangements)  because  time  differences  increase  the  collaborative  work  day  length. 

o.  Outsourcing  projects  are  more  successful  when  the  buyer  and  vendor  have  previously  worked  together 
successfully. 

p.  Outsourcing  development  of  software  is  more  successful  when  the  software  vendor  has  a  higher 
process  maturity  (e.g.  SEI  CMM  rating). 

q.  Outsourcing  development  of  software  is  more  successful  when  the  buyer  has  a  higher  process 
maturity  (e.g.  SEI  CMM  rating). 

r.  Outsourcing  development  of  software  is  more  successful  when  the  vendor  has  a  successful  track 
record. 
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(Question  13  Continued)  Based  upon  your  experience,  identify  your  level  of  agreement  with  the  following 
assertions  about  software  development  outsourcing. 

(Put  the  appropriate  number  from  the  scale  in  the  blank  next  to  each  assertion) 

Agreement  Scale 


Strongly  Disagree  Neither  Agree  Nor  Disagree 

1  2  3 

4 

Strongly  Agree 

5 

Goal  and  Expectation  Assertions 

s.  Outsourcing  projects  with  more  aggressive  cost  reduction  goals  are  less  likely  to  be  successful  than 
those  with  more  modest  cost  reduction  goals. 


t.  Outsourcing  projects  with  more  aggressive  cost  reduction  goals  are  more  likely  to  be  successful  than 
-  those  with  more  modest  cost  reduction  goals. 

u.  Outsourcing  projects  with  more  aggressive  schedule  duration  reduction  goals  are  less  likely  to  be 
-  successful  than  those  with  more  modest  schedule  duration  reduction  goals. 

v.  Outsourcing  projects  with  more  aggressive  schedule  duration  reduction  goals  are  more  likely  to  be 
-  successful  than  those  with  more  modest  schedule  duration  reduction  goals. 

Product  Assertions 

w.  Outsourcing  development  of  software  is  more  successful  when  the  system  is  not  complex. 

x.  Outsourcing  development  of  software  is  more  successful  when  the  system  can  be  easily  divided  into 
-  components  (highly  modular). 

Other  Assertions  (please  list) 

-  y- 

z. 

aa. 
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14.  Based  on  your  experience,  identify  your  level  of  agreement  with  each  of  the  following  assertions  about 
which  factors  determine  whether  product  component  outsourcing  will  be  successful. 

(Put  the  appropriate  number  from  the  scale  in  the  blank  next  to  each  assertion) 

Agreement  Scale 

Strongly  Disagree  Neither  Agree  Nor  Disagree  Strongly  Agree 

1  2  3  4  5 

-  a.  Outsourcing  larger  components  is  generally  more  successful  than  outsourcing  smaller  components. 

— —  b.  Outsourcing  smaller  components  is  generally  more  successful  than  outsourcing  larger  components. 

c.  Outsourcing  components  of  highly  modular  products  is  generally  more  successful  than  outsourcing 
components  of  monolithic  products. 

-  d.  Outsourcing  is  more  successful  when  the  interfaces  for  an  outsourced  component  are  well-defined. 

e.  Outsourcing  is  more  successful  when  the  tools  and  languages  used  by  both  in-house  and  vendor 
developers  are  compatible. 

f.  Outsourcing  is  more  successful  when  an  outsourced  component's  requirements  are  well-defined  up¬ 
front. 

g.  Outsourcing  is  more  successful  when  the  vendor  and  buyer  organizations  communicate  well  and 
overcome  administrative  obstacles  to  solve  problems. 

Other  (please  list) 

h. 
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15.  Based  on  your  experience,  identify  your  level  of  agreement  with  each  of  the  following  assertions  about 
which  factors  determine  if  process  component  (development  activity)  outsourcing  will  be  successful. 

(Put  the  appropriate  number  from  the  scale  in  the  blank  next  to  each  assertion) 

Agreement  Scale 

Strongly  Disagree  Neither  Agree  Nor  Disagree  Strongly  Agree 

1  2  3  4  5 

a.  Outsourcing  is  more  successful  when  organizational  interfaces  and  responsibilities  are  well-defined 
than  when  organizational  interfaces  and  responsibilities  are  loosely  defined. 

b.  Outsourcing  is  more  successful  when  organizational  lifecycle  models  (e.g.  prototyping,  spiral, 
waterfall,  incremental)  used  by  both  the  vendor  and  buyer  are  the  same  rather  than  different. 

c.  Outsourcing  is  more  successful  when  tools  and  methods  allow  information  to  flow  easily  between  the 
vendor  and  in-house  organization. 

_ _  d.  Outsourcing  is  more  successful  when  the  vendor's  process  maturity  (e.g.  SEI  CMM  rating)  is  higher. 

e.  Outsourcing  is  more  successful  when  the  in-house  organization's  process  maturity  (e.g.  SEI  CMM 
rating)  is  higher. 

f.  Outsourcing  is  more  successful  when  the  buyer's  and  vendor's  process  maturity  levels  (e.g.  SEI  CMM 
rating)  are  the  same  or  close  than  when  the  ratings  differ  greatly. 

Other  (please  list) 

_  g- 

h. 


16.  Do  you  have  any  general  comments  about  the  survey,  or  software  outsourcing  in  general? 
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Cost  of  Development 


Cost  Consequence  Validation  Table: 


Survey 

# 

Actual 

Response 

Neural 

Network 

NDR 

ODR 

CDR 

Diff. 

NN 

Diff. 

NDR 

Diff. 

ODR 

Diff. 

CDR 

1 

3.00 

4.19 

2.32 

6.34 

4.07 

1.19 

0.68 

3.34 

1.07 

2 

3.00 

1.12 

5.06 

7.36 

4.28 

1.88 

2.06 

4.36 

1.28 

3 

2.00 

3.52 

2.32 

5.61 

2.77 

1.52 

0.32 

3.61 

0.77 

4 

3.00 

4.47 

5.53 

4.95 

4.16 

1.47 

2.53 

1.95 

1.16 

5 

2.00 

4.91 

2.32 

5.42 

4.04 

2.91 

0.32 

3.42 

2.04 

6 

2.00 

0.17 

3.70 

3.57 

1.89 

1.83 

1.70 

1.57 

0.11 

Total  Difference  (Neural  Network): 

10.80 

i 

Total  Difference  (New  Data  Regression): 

7.61 

!  | 

Total  Difference  (Old  Data  Regression): 

18.25 

1  1 

Total  Difference  (Combined  Data  Regression): 

6.43 

Model  Closest  to  Actual  Result: 

NN  -  3rd 

NDR  -  2nd 

ODR  -  4  th 

CDR-  1st 
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Schedule  Duration 


8.00 


7.00 


6.00 


5.00 


1  2  3  4  5  6 

Respondent's  ID 


Schedule  Consequence  Validation  Table: 


Survey 

# 

Actual 

Response 

Neural 

Network 

NDR 

ODR 

CDR 

Diff. 

NN 

Diff. 

NDR 

Diff. 

ODR 

Diff. 

CDR 

1 

5.00 

3.24 

1.06 

4.13 

3.36 

1.76 

3.94 

0.87 

1.64 

2 

3.00 

6.46 

5.02 

7.36 

5.47 

3.46 

2.02 

4.36 

2.47 

3 

2.00 

3.41 

1.06 

3.10 

3.05 

1.41 

0.94 

1.10 

1.05 

4 

4.00 

2.74 

2.41 

2.22 

4.43 

1.26 

1.59 

1.78 

0.43 

5 

5.00 

5.82 

4.24 

5.87 

5.96 

0.82 

0.76 

0.87 

0.96 

6 

2.00 

2.35 

2.21 

3.42 

3.05 

0.35 

0.21 

1.42 

1.05 

Total  Difference  (Neural  Network): 

9.06 

1  1 

Total  Difference  (New  Data  Regression): 

9.46 

i  r~ 

Total  Difference  (Old  Data  Regression): 

10.40 

1  1 

Total  Difference  (Combined  Data  Regression): 

7.60 

Model  Closest  to  Actual  Result: 

NN  -  2nd 

NDR  -  3rd 

ODR  -  4lh 

CDR  -  1st 
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Intellectual  Capital 


I  □  Actual  Response 

■  Neural  Network 

□  NDR 

□  ODR 
[■CDR 


Respondent's  ID 


Intellectual  Capital  Consequence  Validation  Table: 


Survey 

# 

Actual 

Response 

Neural 

Network 

NDR 

ODR 

CDR 

Diff. 

NN 

Diff. 

NDR 

Diff. 

ODR 

Diff. 

CDR 

1 

4.00 

3.80 

5.04 

3.98 

3.60 

0.20 

1.04 

0.02 

0.40 

2 

4.00 

3.84 

4.20 

7.65 

4.44 

0.16 

0.20 

3.65 

0.44 

3 

3.00 

3.82 

5.04 

4.57 

4.73 

0.82 

2.04 

1.57 

1.73 

4 

4.00 

4.06 

4.16 

5.95 

3.99 

0.06 

0.16 

1.95 

0.01 

5 

5.00 

3.79 

3.38 

5.07 

3.60 

1.21 

1.62 

0.07 

1.40 

6 

3.00 

5.02 

5.56 

4.27 

4.69 

2.02 

2.56 

1.27 

1.69 

Total  Difference  (Neural  Network): 

4.47 

1  1 

Total  Difference  (New  Data  Regression): 

7.62 

i 

Total  Difference  (Old  Data  Regression): 

8.53 

1  1 

Total  Difference  (Combined  Data  Regression): 

5.67 

Model  Closest  to  Actual  Result: 


NN-  1st 
NDR  -  3rd 
ODR  -  4th 
CDR -2nd 


Appendix  H 


Schedule  Flexibility 


Schedule  Flexibility  Consequence  Validation  Table: 


Survey 

# 

Actual 

Response 

Neural 

Network 

NDR 

ODR 

CDR 

Diff. 

NN 

Diff. 

NDR 

Diff. 

ODR 

Diff. 

CDR 

1 

4.00 

6.06 

8.09 

5.44 

5.23 

2.06 

4.09 

1.44 

1.23 

2 

2.00 

1.97 

0.67 

3.81 

3.22 

0.03 

1.33 

1.81 

1.22 

3 

6.00 

5.86 

9.34 

5.71 

5.23 

0.14 

3.34 

0.29 

0.77 

4 

4.00 

5.88 

5.67 

4.84 

4.14 

1.88 

1.67 

0.84 

0.14 

5 

6.00 

4.28 

5.07 

3.71 

3.15 

1.72 

0.93 

2.29 

2.85 

6 

6.00 

4.57 

7.42 

4.19 

5.12 

1.43 

1.42 

1.81 

0.88 

Total  Difference  (Neural  Network): 

7.26 

1  1 

Total  Difference  (New  Data  Regression): 

12.78 

i  |— 

Total  Difference  (Old  Data  Regression): 

8.48 

1  1 

Total  Difference  (Combined  Data  Regression): 

7.09 

Model  Closest  to  Actual  Result: 

NN  -  2nd 

NDR  -  4lh 

ODR  -  3rd 

CDR  -  1st 
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Administrative  Overhead 


Respondent's  ID 


Administrative  Overhead  Consequence  Validation  Table: 


Survey 

# 

Actual 

Response 

Neural 

Network 

NDR 

ODR 

CDR 

Diff. 

NN 

Diff. 

NDR 

Diff. 

ODR 

Diff. 

CDR 

1 

5.00 

4.13 

4.38 

4.39 

4.20 

0.87 

0.62 

0.61 

0.80 

2 

6.00 

3.12 

1.83 

8.25 

2.21 

2.88 

4.17 

2.25 

3.79 

3 

2.00 

4.50 

3.05 

5.38 

4.20 

2.50 

1.05 

3.38 

2.20 

4 

3.00 

4.00 

3.14 

3.70 

3.74 

1.00 

0.14 

0.70 

0.74 

5 

5.00 

4.63 

3.05 

6.70 

5.36 

0.37 

1.95 

1.70 

0.36 

6 

2.00 

2.70 

1.28 

5.65 

2.21 

0.70 

0.72 

3.65 

0.21 

Total  Difference  (Neural  Network): 

8.32 

1  1 

Total  Difference  (New  Data  Regression): 

8.65 

i 

Total  Difference  (Old  Data  Regression): 

12.29 

1  1 

Total  Difference  (Combined  Data  Regression): 

8.10 

Model  Closest  to  Actual  Result: 

NN  -  2nd 

NDR  -  3rd 

ODR  -  4  th 

CDR-  1st 


Appendix  H 


Control  of  Development  Process 


□Actual  Response 

■  Neural  Network 

□  NDR 

□  ODR 

■  CDR 


Respondent's  ID 


Control  Process  Consequence  Validation  Table: 


Survey 

# 

Actual 

Response 

Neural 

Network 

NDR 

ODR 

CDR 

Diff. 

NN 

Diff. 

NDR 

Diff. 

ODR 

Diff. 

CDR 

1 

3.00 

3.47 

2.82 

3.85 

4.03 

0.47 

0.18 

0.85 

1.03 

2 

6.00 

3.94 

4.59 

4.14 

3.30 

2.06 

1.41 

1.86 

2.70 

3 

5.00 

4.23 

3.95 

4.71 

6.24 

0.77 

1.05 

0.29 

1.24 

4 

3.00 

4.76 

3.30 

3.41 

4.07 

1.76 

0.30 

0.41 

1.07 

5 

3.00 

5.00 

4.93 

3.85 

3.26 

2.00 

1.93 

0.85 

0.26 

6 

4.00 

3.37 

1.65 

4.71 

4.19 

0.63 

2.35 

0.71 

0.19 

Total  Difference  (Neural  Network): 

7.69 

Total  Difference  (New  Data  Regression): 

7.22 

Total  Difference  (Old  Data  Regression): 

4.97 

Total  Difference  (Combined  Data  Regression): 

6.49 

Model  Closest  to  Actual  Result: 

NN  -  4lh 

NDR  -  3rd 

ODR  -  1st 

CDR -2nd 
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In-House  Non-Core  Tasks 


7.00 


6.00 


5.00 


4.00 


3.00 


!□  Actual  Response 

■  Neural  Network 

□  NDR 

□  ODR 
[■CDR 


2 


3 


Respondent's  ID 


In-house  Non-Core  Consequence  Validation  Table: 


Survey 

# 

Actual 

Response 

Neural 

Network 

NDR 

ODR 

CDR 

Diff. 

NN 

Diff. 

NDR 

Diff. 

ODR 

Diff. 

CDR 

1 

5.00 

3.54 

2.82 

4.50 

4.74 

1.46 

2.18 

0.50 

0.26 

2 

2.00 

5.05 

6.05 

4.84 

3.71 

3.05 

4.05 

2.84 

1.71 

3 

2.00 

4.11 

1.76 

4.50 

4.42 

2.11 

0.24 

2.50 

2.42 

4 

4.00 

3.92 

2.84 

4.33 

2.63 

0.08 

1.16 

0.33 

1.37 

5 

2.00 

2.90 

3.82 

2.43 

2.40 

0.90 

1.82 

0.43 

0.40 

6 

2.00 

1.43 

1.76 

2.50 

1.45 

0.57 

0.24 

0.50 

0.55 

Total  Difference  (Neural  Network): 

8.17 

1  1 

Total  Difference  (New  Data  Regression): 

9.69 

Total  Difference  (Old  Data  Regression): 

7.10 

1  1 

Total  Difference  (Combined  Data  Regression): 

6.71 

Model  Closest  to  Actual  Result: 


NN  -  3rd 
NDR  -  4th 
ODR  -  2nd 
CDR  -  1st 


Appendix  H 


In-House  Personnel  Turnover 


7.00 


6.00 


5.00 


1  2  3  4  5  6 


□Actual  Response 

■  Neural  Network 

□  NDR 

□  ODR 

■  CDR 


Respondent's  ID 


In-house  Personnel  Turnover  Consequence  Validation  Table: 


Survey 

# 

Actual 

Response 

Neural 

Network 

NDR 

ODR 

CDR 

Diff. 

NN 

Diff. 

NDR 

Diff. 

ODR 

Diff. 

CDR 

1 

3.00 

4.10 

2.34 

4.34 

3.75 

1.10 

0.66 

1.34 

0.75 

2 

2.00 

6.22 

6.38 

3.20 

4.17 

4.22 

4.38 

1.20 

2.17 

3 

2.00 

4.15 

1.65 

4.56 

3.75 

2.15 

0.35 

2.56 

1.75 

4 

3.00 

3.12 

3.54 

3.42 

3.86 

0.12 

0.54 

0.42 

0.86 

5 

4.00 

4.08 

3.40 

3.10 

3.75 

0.08 

0.60 

0.90 

0.25 

6 

5.00 

1.79 

1.65 

3.96 

3.75 

3.21 

3.35 

1.04 

1.25 

Total  Difference  (Neural  Network): 

10.88 

Total  Difference  (New  Data  Regression): 

9.88 

Total  Difference  (Old  Data  Regression): 

7.46 

Total  Difference  (Combined  Data  Regression): 

7.03 

Model  Closest  to  Actual  Result: 

NN  -  4lh 

NDR  -  3rd 

ODR  -  2nd 

CDR  -  1st 
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Learning  Curve 


- 

■ 

- 

1  2  3  4  5  6 


!□  Actual  Response 

■  Neural  Network 

□  NDR 

□  ODR 

■  CDR 


Respondent's  ID 


Learning  Curve  Consequence  Validation  Table: 


Survey 

# 

Actual 

Response 

Neural 

Network 

NDR 

ODR 

CDR 

Diff. 

NN 

Diff. 

NDR 

Diff. 

ODR 

Diff. 

CDR 

1 

5.00 

4.52 

4.44 

3.91 

4.28 

0.48 

0.56 

1.09 

0.72 

2 

3.00 

4.71 

3.79 

3.91 

3.83 

1.71 

0.79 

0.91 

0.83 

3 

5.00 

3.56 

5.08 

3.91 

4.28 

1.44 

0.08 

1.09 

0.72 

4 

5.00 

3.98 

5.73 

5.33 

5.29 

1.02 

0.73 

0.33 

0.29 

5 

6.00 

3.99 

4.44 

4.27 

4.28 

2.01 

1.56 

1.73 

1.72 

6 

5.00 

3.30 

4.44 

3.91 

3.83 

1.70 

0.56 

1.09 

1.17 

Total  Difference  (Neural  Network): 

8.36 

1  1 

Total  Difference  (New  Data  Regression): 

4.28 

Total  Difference  (Old  Data  Regression): 

6.24 

1  1 

Total  Difference  (Combined  Data  Regression): 

5.45 

Model  Closest  to  Actual  Result: 

NN-4lh 

NDR  -  1st 

ODR  -  3rd 

CDR -2nd 


Appendix  H 


Project  Risks 


Risk  Consequence  Validation  Table: 


Survey 

# 

Actual 

Response 

Neural 

Network 

NDR 

ODR 

CDR 

Diff. 

NN 

Diff. 

NDR 

Diff. 

ODR 

Diff. 

CDR 

1 

5.00 

4.17 

3.48 

4.52 

4.42 

0.83 

1.52 

0.48 

0.58 

2 

2.00 

4.30 

4.49 

4.52 

3.67 

2.30 

2.49 

2.52 

1.67 

3 

3.00 

3.95 

3.48 

4.52 

4.42 

0.95 

0.48 

1.52 

1.42 

4 

3.00 

2.27 

3.51 

2.50 

0.97 

0.73 

0.51 

0.50 

2.03 

5 

5.00 

2.98 

3.48 

4.52 

3.53 

2.02 

1.52 

0.48 

1.47 

6 

4.00 

3.30 

3.48 

4.52 

2.64 

0.70 

0.52 

0.52 

1.36 

Total  Difference  (Neural  Network): 

7.53 

Total  Difference  (New  Data  Regression): 

7.04 

Total  Difference  (Old  Data  Regression): 

6.02 

Total  Difference  (Combined  Data  Regression): 

8.53 

Model  Closest  to  Actual  Result: 

NN  -  3rd 

NDR  -  2nd 

ODR  -  1st 

CDR -4"’ 
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Quality 


□Actual  Response 

■  Neural  Network 

□  NDR 

□  ODR 

■  CDR 


Respondent's  ID 


Quality  Consequence  Validation  Table: 


Survey 

# 

Actual 

Response 

Neural 

Network 

NDR 

ODR 

CDR 

Diff. 

NN 

Diff. 

NDR 

Diff. 

ODR 

Diff. 

CDR 

1 

3.00 

4.51 

5.09 

4.55 

4.58 

1.51 

2.09 

1.55 

1.58 

2 

1.00 

6.00 

4.23 

5.70 

5.49 

5.00 

3.23 

4.70 

4.49 

3 

6.00 

5.50 

4.20 

4.55 

4.58 

0.50 

1.80 

1.45 

1.42 

4 

6.00 

5.69 

3.99 

3.86 

5.06 

0.31 

2.01 

2.14 

0.94 

5 

3.00 

4.99 

5.09 

4.55 

4.58 

1.99 

2.09 

1.55 

1.58 

6 

6.00 

5.31 

5.04 

4.55 

6.07 

0.69 

0.96 

1.45 

0.07 

Total  Difference  (Neural  Network): 

10.00 

1  1 

Total  Difference  (New  Data  Regression): 

12.18 

Total  Difference  (Old  Data  Regression): 

12.84 

1  1 

Total  Difference  (Combined  Data  Regression): 

10.08 

Model  Closest  to  Actual  Result: 


NN-  1st 
NDR  -  3rd 
ODR  -  4th 
CDR -2nd 


Appendix  H 


Rework 


□Actual  Response 

■  Neural  Network 

□  NDR 

□  ODR 

■  CDR 


Respondent's  ID 


Rework  Consequence  Validation  Table: 


Survey 

# 

Actual 

Response 

Neural 

Network 

NDR 

ODR 

CDR 

Diff. 

NN 

Diff. 

NDR 

Diff. 

ODR 

Diff. 

CDR 

1 

5.00 

4.92 

3.83 

4.07 

4.22 

0.08 

1.17 

0.93 

0.78 

2 

5.00 

4.87 

5.41 

3.03 

5.16 

0.13 

0.41 

1.97 

0.16 

3 

5.00 

5.18 

4.61 

4.07 

3.70 

0.18 

0.39 

0.93 

1.30 

4 

4.00 

4.37 

3.00 

3.37 

3.23 

0.37 

1.00 

0.63 

0.77 

5 

5.00 

6.02 

5.50 

4.07 

4.22 

1.02 

0.50 

0.93 

0.78 

6 

2.00 

3.62 

3.83 

5.20 

3.70 

1.62 

1.83 

3.20 

1.70 

Total  Difference  (Neural  Network): 

3.40 

Total  Difference  (New  Data  Regression): 

5.30 

Total  Difference  (Old  Data  Regression): 

8.59 

Total  Difference  (Combined  Data  Regression): 

5.49 

Model  Closest  to  Actual  Result: 


NN  -  1st 
NDR  -  2nd 
ODR  -  4lh 
CDR -3rd 
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Project  Visibility 


Visibility  Consequence  Validation  Table: 


Survey 

# 

Actual 

Response 

Neural 

Network 

NDR 

ODR 

CDR 

Diff. 

NN 

Diff. 

NDR 

Diff. 

ODR 

Diff. 

CDR 

1 

4.00 

5.16 

4.21 

3.72 

3.27 

1.16 

0.21 

0.28 

0.73 

2 

3.00 

8.32 

9.51 

6.20 

7.05 

5.32 

6.51 

3.20 

4.05 

3 

5.00 

5.38 

3.37 

4.28 

3.97 

0.38 

1.63 

0.72 

1.03 

4 

5.00 

4.08 

0.42 

5.97 

4.30 

0.92 

4.58 

0.97 

0.70 

5 

4.00 

3.29 

3.46 

3.04 

3.82 

0.71 

0.54 

0.96 

0.18 

6 

6.00 

4.41 

5.98 

5.96 

5.05 

1.59 

0.02 

0.04 

0.95 

Total  Difference  (Neural  Network): 

10.08 

1  1 

Total  Difference  (New  Data  Regression): 

13.49 

Total  Difference  (Old  Data  Regression): 

6.17 

1  1 

Total  Difference  (Combined  Data  Regression): 

7.64 

Model  Closest  to  Actual  Result: 


NN  -  3rd 
NDR  -  4  th 
ODR  -  1st 
CDR  —  2nd 


Appendix  H 


Control  of  Product 


□Actual  Response 

■  Neural  Network 

□  NDR 

□  ODR 

■  CDR 


Respondent's  ID 


Control  Product  Consequence  Validation  Table: 


Survey 

# 

Actual 

Response 

Neural 

Network 

NDR 

ODR 

CDR 

Diff. 

NN 

Diff. 

NDR 

Diff. 

ODR 

Diff. 

CDR 

1 

4.00 

3.43 

4.48 

4.19 

3.95 

0.57 

0.48 

0.19 

0.05 

2 

3.00 

5.55 

6.18 

5.53 

5.96 

2.55 

3.18 

2.53 

2.96 

3 

6.00 

5.07 

3.19 

6.00 

4.78 

0.93 

2.81 

0.00 

1.22 

4 

6.00 

3.56 

5.43 

1.95 

3.51 

2.44 

0.57 

4.05 

2.49 

5 

3.00 

2.60 

3.19 

4.19 

3.95 

0.40 

0.19 

1.19 

0.95 

6 

6.00 

4.88 

4.89 

4.19 

4.19 

1.12 

1.11 

1.81 

1.81 

Total  Difference  (Neural  Network): 

8.01 

Total  Difference  (New  Data  Regression): 

8.34 

Total  Difference  (Old  Data  Regression): 

9.77 

Total  Difference  (Combined  Data  Regression): 

9.48 

Model  Closest  to  Actual  Result: 


NN  -  1st 
NDR  -  2nd 
ODR  -  4lh 
CDR -2nd 
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Cost  of  Project  Changes 


8.00 


7.00 


6.00 


1  2  3  4  5  6 

Respondent's  ID 


!□  Actual  Response 

■  Neural  Network 

□  NDR 

□  ODR 

■  CDR 


Change  Cost  Consequence  Validation  Table: 


Survey 

# 

Actual 

Response 

Neural 

Network 

NDR 

ODR 

CDR 

Diff. 

NN 

Diff. 

NDR 

Diff. 

ODR 

Diff. 

CDR 

1 

3.00 

4.97 

3.51 

5.41 

4.59 

1.97 

0.51 

2.41 

1.59 

2 

2.00 

3.34 

2.14 

5.55 

3.49 

1.34 

0.14 

3.55 

1.49 

3 

2.00 

1.90 

3.51 

5.41 

3.47 

0.10 

1.51 

3.41 

1.47 

4 

3.00 

6.26 

4.80 

6.83 

5.51 

3.26 

1.80 

3.83 

2.51 

5 

2.00 

4.57 

3.51 

4.93 

5.31 

2.57 

1.51 

2.93 

3.31 

6 

2.00 

2.53 

1.43 

5.46 

1.61 

0.53 

0.57 

3.46 

0.39 

Total  Difference  (Neural  Network): 

9.77 

1  1 

Total  Difference  (New  Data  Regression): 

6.04 

Total  Difference  (Old  Data  Regression): 

19.59 

1  1 

Total  Difference  (Combined  Data  Regression): 

10.76 

Model  Closest  to  Actual  Result: 

NN  -  2nd 

NDR  -  1st 

ODR  -  4th 

CDR -3rd 


Appendix  H 


Language/Culture/Distance  Problems 


□Actual  Response 

■  Neural  Network 

□  NDR 

□  ODR 

■  CDR 


Respondent's  ID 


Language/Culture/Distance  Problems  Consequence  Validation  Table: 


Survey 

# 

Actual 

Response 

Neural 

Network 

NDR 

ODR 

CDR 

Diff. 

NN 

Diff. 

NDR 

Diff. 

ODR 

Diff. 

CDR 

1 

5.00 

4.43 

4.19 

4.78 

4.52 

0.57 

0.81 

0.22 

0.48 

2 

3.00 

4.83 

2.71 

4.34 

3.91 

1.83 

0.29 

1.34 

0.91 

3 

5.00 

4.06 

3.38 

4.51 

4.01 

0.94 

1.62 

0.49 

0.99 

4 

5.00 

5.13 

2.74 

4.97 

4.40 

0.13 

2.26 

0.03 

0.60 

5 

6.00 

3.98 

4.19 

4.94 

4.84 

2.02 

1.81 

1.06 

1.16 

6 

5.00 

4.51 

3.72 

4.07 

4.11 

0.49 

1.28 

0.93 

0.89 

Total  Difference  (Neural  Network): 

5.98 

Total  Difference  (New  Data  Regression): 

8.07 

Total  Difference  (Old  Data  Regression): 

4.07 

Total  Difference  (Combined  Data  Regression): 

5.03 

Model  Closest  to  Actual  Result: 

NN  -  3rd 

NDR  -  4lh 

ODR  -  1st 

CDR -2nd 
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Turf  War  (Teamwork)  Problems 


Respondent's  ID 


Turf  War  (Teamwork)  Problems  Consequence  Validation  Table: 


Survey 

# 

Actual 

Response 

Neural 

Network 

NDR 

ODR 

CDR 

Diff. 

NN 

Diff. 

NDR 

Diff. 

ODR 

Diff. 

CDR 

1 

5.00 

5.40 

3.80 

4.62 

5.36 

0.40 

1.20 

0.38 

0.36 

2 

4.00 

4.15 

3.79 

2.94 

5.41 

0.15 

0.21 

1.06 

1.41 

3 

3.00 

4.20 

5.11 

3.16 

3.67 

1.20 

2.11 

0.16 

0.67 

4 

5.00 

5.78 

2.56 

5.20 

4.72 

0.78 

2.44 

0.20 

0.28 

5 

5.00 

4.50 

3.80 

3.12 

4.02 

0.50 

1.20 

1.88 

0.98 

6 

5.00 

3.87 

2.94 

5.60 

4.02 

1.13 

2.06 

0.60 

0.98 

Total  Difference  (Neural  Network): 

4.16 

l  l 

Total  Difference  (New  Data  Regression): 

9.22 

Total  Difference  (Old  Data  Regression): 

4.28 

1  1 

Total  Difference  (Combined  Data  Regression): 

4.68 

Model  Closest  to  Actual  Result: 


NN-  1st 
NDR  -  4  th 
ODR  -  2  nd 
CDR  —  3rd 
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Failure  Likelihood 


Failure  Likelihood  Consequence  Validation  Table: 


Survey 

# 

Actual 

Response 

Neural 

Network 

NDR 

ODR 

CDR 

Diff. 

NN 

Diff. 

NDR 

Diff. 

ODR 

Diff. 

CDR 

1 

4.00 

4.09 

3.94 

3.15 

4.18 

0.09 

0.06 

0.85 

0.18 

2 

1.00 

3.20 

5.85 

2.25 

5.49 

2.20 

4.85 

1.25 

4.49 

3 

2.00 

2.56 

3.29 

3.85 

3.65 

0.56 

1.29 

1.85 

1.65 

4 

4.00 

5.86 

2.28 

4.35 

4.19 

1.86 

1.72 

0.35 

0.19 

5 

3.00 

3.94 

3.69 

3.85 

4.18 

0.94 

0.69 

0.85 

1.18 

6 

3.00 

2.89 

4.58 

5.43 

3.74 

0.11 

1.58 

2.43 

0.74 

Total  Difference  (Neural  Network): 

5.76 

Total  Difference  (New  Data  Regression): 

10.19 

Total  Difference  (Old  Data  Regression): 

7.58 

Total  Difference  (Combined  Data  Regression): 

8.43 

Model  Closest  to  Actual  Result: 


NN  -  1st 
NDR  -  4lh 
ODR  -  2nd 
CDR -3rd 
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Response  to  Customer 


8.00 


7.00 


1  2  3  4  5  6 


Response  to  Customer  Consequence  Validation  Table: 


Survey 

# 

Actual 

Response 

Neural 

Network 

NDR 

ODR 

CDR 

Diff. 

NN 

Diff. 

NDR 

Diff. 

ODR 

Diff. 

CDR 

1 

4.00 

4.72 

6.87 

4.60 

4.41 

0.72 

2.87 

0.60 

0.41 

2 

4.00 

4.17 

3.80 

3.90 

5.11 

0.17 

0.20 

0.10 

1.11 

3 

5.00 

4.21 

5.64 

3.38 

4.44 

0.79 

0.64 

1.62 

0.56 

4 

5.00 

3.13 

3.56 

4.33 

3.96 

1.87 

1.44 

0.67 

1.04 

5 

6.00 

4.60 

3.66 

4.20 

3.70 

1.40 

2.34 

1.80 

2.30 

6 

6.00 

5.68 

6.28 

5.12 

4.91 

0.32 

0.28 

0.88 

1.09 

Total  Difference  (Neural  Network): 

5.27 

1  1 

Total  Difference  (New  Data  Regression): 

7.77 

Total  Difference  (Old  Data  Regression): 

5.67 

1  1 

Total  Difference  (Combined  Data  Regression): 

6.51 

Model  Closest  to  Actual  Result: 


NN-  1st 
NDR  -  4  th 
ODR  —  2nd 
CDR  —  3rd 
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Response  to  Organizati< 


□Actual  Response 

■  Neural  Network 

□  NDR 

□  ODR 

■  CDR 


Respondent's  ID 


Response  to  Organization  Consequence  Validation  Table: 


Survey 

# 

Actual 

Response 

Neural 

Network 

NDR 

ODR 

CDR 

Diff. 

NN 

Diff. 

NDR 

Diff. 

ODR 

Diff. 

CDR 

1 

4.00 

3.92 

3.79 

5.07 

3.60 

0.08 

0.21 

1.07 

0.40 

2 

2.00 

2.04 

3.65 

3.14 

2.40 

0.04 

1.65 

1.14 

0.40 

3 

6.00 

4.54 

4.94 

3.80 

4.11 

1.46 

1.06 

2.20 

1.89 

4 

5.00 

4.61 

6.17 

4.30 

6.30 

0.39 

1.17 

0.70 

1.30 

5 

5.00 

4.78 

3.92 

4.33 

4.11 

0.22 

1.08 

0.67 

0.89 

6 

6.00 

4.35 

4.80 

5.07 

4.62 

1.65 

1.20 

0.93 

1.38 

Total  Difference  (Neural  Network): 

3.84 

Total  Difference  (New  Data  Regression): 

6.37 

Total  Difference  (Old  Data  Regression): 

6.71 

Total  Difference  (Combined  Data  Regression): 

6.26 

Model  Closest  to  Actual  Result: 


NN  -  1st 
NDR  -  3rd 
ODR  -  4lh 
CDR -2nd 
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Validation  Summary 


NN 

NDR 

ODR 

CDR 

148.81 

171.2 

175.01 

143.46 

2nd 

3rd 

4th 

1st 

Though  Combined  Data  Regression  (CDR)  learning  mechanism  was  closest  in  predicting 
the  actual  response,  Neural  Networks  (NN)  almost  tied  CDR  by  only  5.35  points.  The 
NN  and  CDR  performance  results  were  reasonable  due  to  both  being  trained  with  all  the 
data. 

There  was  a  tight  correlating  coupling  between  all  four  learning  mechanisms  with  a  few 
exceptions.  In  these  exceptions,  New  Data  Regression  (NDR)  and  Old  Data  Regression 
(ODR),  found  problems  with  interpolation  which  would  cause  an  extreme  output  greater 
than  seven  or  less  than  one.  This  was  due  to  by  not  having  enough  data  in  either  case 
which  was  the  reason  why  the  CDR  was  included. 
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Appendix  I  -  SODS2  Window  Outline 


1.1  SODS2  Introductory  Window 

With  the  MS  Access  VB  development  tool,  an  introductory  window,  found  in 
Figure  1  below,  provided  the  user  a  short  description  of  SODS2  along  with  a  disclaimer 
that  such  an  observational  tool  should  not  be  mistaken  for  an  experimental  cause  and 
effect  tool.  This  application  was  built  to  provide  observational  type  knowledge.  Along 
with  this  user’s  introductory  guidance,  SODS2  used  an  immediate  startup  module.  Upon 
the  user  opening  the  outsourcing  MS  Access  database,  the  introductory  window  would 
immediately  be  opened  and  maximized  to  enforce  a  certain  flow  for  this  and  the  other 
windows  to  follow.  Along  with  this  maximizing  feature,  the  window  control  buttons 
normally  found  on  the  top  right  comer  were  turned  off  preventing  the  accidental  bypass 
of  such  an  orderly  window  flow.  This  provided  SODS2  a  user  friendly  approach.  Once 
the  introductory  window  has  been  displayed  for  the  very  first  time,  the  user  could  select 
to  turn  it  off  and  immediately  proceed  to  the  maximize  version  of  the  main  window. 
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Figure  1  SODS2  Introductory  Window 
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1.2  SODS2  Main  Window 


The  main  window,  found  below  in  Figure  2,  provided  the  user  an  easy  “point  and 
click”  interface  for  all  required  input.  Help  labels  were  added  to  guide  the  user  through 
each  color  coded  input  division.  Smart  help  tags  were  also  added  so  additional 
information  regarding  each  input  and  command  button  would  be  displayed  as  needed. 
This  window  also  would  open  maximized  to  provide  an  orderly  SODS2  window  flow. 
Four  command  buttons  were  found  on  this  window.  Clear  selected  input  command 
button  allowed  the  user  to  wipe  away  all  selections.  Close  SODS2  application  command 
button  allowed  the  user  to  exit  the  application  and  provided  normal  access  to  the  database 
contained  within  the  application.  NN  report  command  button  took  the  user’s  inputs  and 
proceeded  to  opening  a  maximized  NN  report  window.  Regression  report  command 
button  took  the  user’s  inputs  and  proceeded  to  opening  a  maximized  regression  report 
window. 
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Figure  2  SODS2  Main  Window 
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I  Step  1:  Select  domain  lor  (he 


1.3  SODS2  Regression  Report  Window 


Upon  the  regression  report  command  button  being  clicked,  a  regression  report 
window,  found  below  in  Figure  3,  would  be  opened  fully  maximized  with  all  right  top 
comer  window  commands  turned  off.  Upon  SODS2  opening  this  window,  all 
information  relating  to  the  regression  report  was  transferred  so  that  regression  results  can 
be  correctly  displayed,  printed  and/or  saved.  Each  of  these  functions  was  found  coded 
behind  their  respective  command  buttons.  Before  printing  or  saving  such  report,  the  user 
was  given  edit  capabilities  making  the  versatility  of  such  a  report  much  more  dynamic. 
The  logic  behind  the  results  also  provided  the  user  an  easy  to  use  application.  Based 
upon  the  regression  model  report  command  button  (located  in  the  main  window)  being 
clicked,  SODS2  would  decipher  the  user’s  inputs  and  selected  consequences  to  generate 
the  associated  regression  model  output.  Such  output  would  use  the  regression  logic  to 
determine  whether  the  proposed  outsourcing  plan  was  favorable  or  unfavorable.  This 
output  decision  came  complete  with  better/worse  scale.  Any  consequence  that  this  logic 
found  as  “dramatically  better”  received  “+++,”  “better”  received  “++,”  “slightly  better” 
received  “+,”  “neutral  or  no  change”  received  nothing,  “slightly  worse”  received 
“worse”  received  or  “dramatically  worse”  received  “ — .”  All  inputs  and  outputs 
were  then  displayed  in  an  orderly  fashion  within  the  main  text  box.  From  this  window 
the  user  was  given  a  command  button  to  allow  a  review  of  the  corresponding  assertions 
(those  methods  found  to  increase  outsourcing  success).  Once  this  button  was  clicked,  the 
application  would  open  that  report  window.  Once  the  user  finished  viewing  the 
regression  report  window,  a  close  button  provided  the  means  to  exit  this  window  taking 
the  user  back  to  the  main  window. 
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Figure  3  SODS2  Regression  Report  Window 


SI  Regression  Report  Window 


Report  of  Outsourcing  Consequences  using  Linear  Regression  Formulas 


Selected  Outsourcing  Consequences  Include: 

Schedule 
Control  Process 
Quality 

Control  Product 
Change  Cost 

Selected  Domains  Include: 

Systems  Communications 
Shrink  Business 
Enterprise  Manufacturing 
Component  Operating  System 

Selected  Outsourcing  Process  Strategies  Include: 

Requirements 

Maintenance 

Software  Engineer  Support 
Configuration  Management 

Selected  Outsourcing  Product  Strategies  Include: 

Common  Customizable 

New  Data  Calculated  Consequences: 

New  Data  Schedule  Consequence  Equals  3.31 
Slightly  Reduces  This  Consequence  (This  is  Slightly  Better  +) 

New  Data  Control  Process  Consequence  Equals  2.58 
Slightly  Reduces  This  Consequence  (This  is  Slightly  Worse  -) 

New  Data  Quality  Consequence  Equals  4.43 
T  his  Consequence  I  s  N  eutral  (N  either  B  etter  or  Worse) 

New  Data  Control  Product  Consequence  Equals  5.94 
Significantly  Increases  This  Consequence  (This  is  Better  ++) 

New  Data  Change  Cost  Consequence  Equals  5.51 
Significantly  Increases  This  Consequence  (This  is  Worse  --) 

Old  Data  Calculated  Consequences: 

Old  Data  Schedule  Consequence  Equals  3.04 
Slightly  Reduces  This  Consequence  (This  is  Slightly  Better  +) 

Old  Data  Control  Process  Consequence  Equals  4.25 
T  his  Consequence  I  s  N  eutral  (N  either  B  etter  or  Worse) 

Old  Data  Quality  Consequence  Equals  3.92 
T  his  Consequence  I  s  N  eutral  (N  either  B  etter  or  Worse) 

Old  Data  Control  Product  Consequence  Equals  3.07 
Slightly  Reduces  This  Consequence  (This  is  Slightly  Worse  -) 

Old  Data  Change  Cost  Consequence  Equals  7.6 
Dramatically  Increases  This  Consequence  (This  is  Dramatically  Worse  — ) 

Combined  Data  Calculated  Consequences: 

Combined  Data  Schedule  Conseauence  Eauals  2.48 

Not  Satisfied  with  the  Resulting 


Print  Report 


Save  Report 


Consequences  -  Click  Assertion 
Button  to  Discover  Ways  for 
Improving  Outsourcing  Results 
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1.4  SODS2  NN  Report  Window 


Upon  the  NN  report  command  button  being  clicked  from  the  main  window,  a  NN 
report  window,  found  below  in  Figure  4,  would  be  opened  fully  maximized  with  all  right 
top  comer  window  commands  turned  off.  This  window  duplicated  all  the  same  features 
found  in  the  regression  report  window  with  the  exception  that  the  results  were  calculated 
with  the  corresponding  NN  consequence  DLLs. 
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Figure  4  SODS2  NN  Report  Window 


Page  4 


1.5  SODS2  Assertion  Report  Window 


From  both  the  regression  or  NN  report  windows,  the  user  was  given  a  command 
button  that  would  allow  them  to  view  this  assertion  report  illustrated  in  Figure  5.  This 
report  was  very  useful  when  the  tool  indicated  worse  than  expected  consequence  results 
and  the  user  was  limited  in  making  strategy  changes.  Thus,  these  assertions  reported 
within  this  window  would  guide  the  user  to  make  the  best  of  the  given  situation.  This 
report  like  the  others  would  open  maximized  with  the  window  control  button  at  the  top 
right  comer  turned  off.  The  report  also  used  logic  to  decide  which  assertions  were 
appropriate  given  the  user’s  input.  Again,  like  the  other  report  windows,  this  information 
was  editable,  printable  and  savable. 
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Figure  5  SODS2  Assertion  Report  Window 


81  Regression  Report  Window 


_ Report  of  Assertions  Matching  Outsourcing  Strategy  to  Improve  Consequences _ 

Assertions  report  includes  techniques  recorded  by  the  majority  of  respondents  that  improve  outsourcing. 

Standard  Outsourcing  Project  Assertions  Include: 

Contractor  needs  experience  with  projects  coding  language  and  associated  tools 

Contractor  needs  experience  with  projects  domain 

Inhouse  buyers  organization  needs  experience  with  projects  domain 

It  is  good  when  buyer  realizing  different  domains  have  different  effects  on  outsourcing 

Contractors  availability  (A  vs.  B  team,  how  busy  are  they,  and  number  employed)  affects  outsourcing  success 

Amount  of  contractors  reusable  designs  and  code  components  tends  to  increase  outsourcing 

Standard  Outsourcing  Relationship  Assertions  Include: 

Contractors  track  record  is  a  good  indicator  for  imroving  outsourcing 
Frequent  reviews  between  buyer  and  contractor  increases  outsourcing  success 

Increasing  buyer  and  contractor  communications  (face  to  face,  phone  calls,  email,  etc...)  increases  outsourcing  success 

Increasing  buyers  visibility  into  contractors  development  process  increases  outsourcing  success 

Having  payment  incentives  (payment  strategy)  increases  outsourcing  success 

Having  a  past,  successful  working  relationship  increases  outsourcing  success 

Contractors  process  maturity  level  is  a  good  indicator  for  improving  outsourcing 

Buyers  process  maturity  level  is  a  good  indicator  for  improving  outsourcing 

Process  Outsourcing  Assertions  Include: 

Compatible  tools  and  methods  increasing  the  flow  of  information  increases  outsourcing  success 

Having  interfaces  and  responsibilities  well  defined  increases  outsourcing  success 

Sharing  same  lifecycles  between  buyer  and  contractor  increases  outsourcing  success 

Having  a  higher  contractor  maturity  level  increases  outsourcing  success 

Having  a  higher  buyer  maturity  level  increases  outsourcing  success 

Having  a  similar  maturity  level  between  buyer  and  contractor  increases  outsourcing  success 

Product  Outsourcing  Assertions  Include: 

Increasing  buyer  and  contractor  communication  increases  outsourcing  success 
Having  interfaces  well  defined  increases  outsourcing  success 
Having  requirements  well  defined  increases  outsourcing  success 
Compatible  tools  and  progamming  languages  increases  outsourcing  success 
Having  a  higher  modular  product  increases  outsourcing  success 
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